Jul 09 18:57:18.281463 ip-10-0-128-157 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 09 18:57:18.281478 ip-10-0-128-157 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 09 18:57:18.281487 ip-10-0-128-157 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 09 18:57:18.281771 ip-10-0-128-157 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 09 18:57:28.328278 ip-10-0-128-157 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 09 18:57:28.328293 ip-10-0-128-157 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 12f94bc7e69342cba132f5d4d9941260 --
Jul 09 18:59:15.200143 ip-10-0-128-157 systemd[1]: Starting Kubernetes Kubelet...
Jul 09 18:59:15.581741 ip-10-0-128-157 kubenswrapper[2569]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:59:15.581741 ip-10-0-128-157 kubenswrapper[2569]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 09 18:59:15.581741 ip-10-0-128-157 kubenswrapper[2569]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:59:15.581741 ip-10-0-128-157 kubenswrapper[2569]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 09 18:59:15.581741 ip-10-0-128-157 kubenswrapper[2569]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 09 18:59:15.583610 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.583525    2569 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 09 18:59:15.586963 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586948    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:59:15.586963 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586964    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586968    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586973    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586977    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586980    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586983    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586986    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586989    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586992    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586994    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586997    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.586999    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587003    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587006    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587009    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587012    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587014    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587017    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587019    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:59:15.587032 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587022    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587029    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587032    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587035    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587038    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587041    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587044    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587046    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587049    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587051    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587054    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587056    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587059    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587062    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587064    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587067    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587069    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587072    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587074    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587077    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:59:15.587469 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587080    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587082    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587084    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587087    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587090    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587092    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587094    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587097    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587099    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587102    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587104    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587107    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587109    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587111    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587115    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587118    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587120    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587123    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587125    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587129    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:59:15.587959 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587131    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587134    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587136    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587138    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587141    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587143    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587146    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587148    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587150    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587153    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587155    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587158    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587160    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587163    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587165    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587168    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587170    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587172    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587174    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587177    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:59:15.588560 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587179    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587182    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587184    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587186    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587189    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587193    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587199    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587203    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587205    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587208    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587210    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587213    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587216    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587218    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587220    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587223    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587225    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587228    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:59:15.589106 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.587230    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:59:15.591702 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591682    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:59:15.591702 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591699    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:59:15.591702 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591705    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591710    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591714    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591717    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591720    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591723    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591726    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591729    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591731    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591734    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591736    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591739    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591742    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591744    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591747    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591750    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591752    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591755    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591758    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591761    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:59:15.591830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591764    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591766    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591768    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591771    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591773    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591776    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591779    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591781    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591784    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591786    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591788    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591791    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591794    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591796    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591800    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591802    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591805    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591807    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591810    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:59:15.592337 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591814    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591818    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591820    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591823    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591825    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591828    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591831    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591834    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591836    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591840    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591843    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591846    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591848    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591851    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591854    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591858    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591862    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591865    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591884    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:59:15.592796 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591887    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591890    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591893    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591895    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591898    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591902    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591904    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591907    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591910    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591913    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591916    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591918    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591921    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591924    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591926    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591929    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591931    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591934    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591936    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591939    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:59:15.593290 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591941    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591944    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591946    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591949    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591952    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591954    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591956    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591959    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591961    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591964    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591966    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591969    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591972    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591974    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591976    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591979    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591981    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591984    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.591986    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592427    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:59:15.593782 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592435    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592439    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592443    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592445    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592448    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592451    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592454    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592456    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592459    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592461    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592464    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592467    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592469    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592472    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592474    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592478    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592480    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592483    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592485    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592488    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:59:15.594315 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592490    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592493    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592495    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592499    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592502    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592504    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592507    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592509    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592512    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592516    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592519    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592522    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592525    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592529    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592531    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592534    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592537    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592539    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592543    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:59:15.594830 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592547    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592550    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592553    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592556    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592558    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592561    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592563    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592566    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592568    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592573    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592575    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592579    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592583    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592585    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592588    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592591    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592594    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592596    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592599    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:59:15.595349 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592601    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592604    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592607    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592609    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592612    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592614    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592616    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592619    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592621    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592624    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592626    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592629    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592631    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592634    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592636    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592638    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592641    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592644    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592646    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:59:15.595793 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592649    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592651    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592653    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592656    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592659    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592661    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592663    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592666    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592669    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592672    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592674    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592677    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592679    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592682    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592684    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592687    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592689    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592692    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592694    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592697    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.592699    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:59:15.596288 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592771    2569 flags.go:64] FLAG: --address="0.0.0.0"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592778    2569 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592784    2569 flags.go:64] FLAG: --anonymous-auth="true"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592790    2569 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592795    2569 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592800    2569 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592807    2569 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592813    2569 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592818    2569 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592822    2569 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592825    2569 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592828    2569 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592831    2569 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592834    2569 flags.go:64] FLAG: --cgroup-root=""
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592837    2569 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592839    2569 flags.go:64] FLAG: --client-ca-file=""
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592843    2569 flags.go:64] FLAG: --cloud-provider="external"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592846    2569 flags.go:64] FLAG: --cluster-dns="[]"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592850    2569 flags.go:64] FLAG: --cluster-domain=""
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592853    2569 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592856    2569 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592859    2569 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592862    2569 flags.go:64] FLAG: --container-log-max-files="5"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592879    2569 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 09 18:59:15.596819 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592883    2569 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592886    2569 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592889    2569 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592892    2569 flags.go:64] FLAG: --contention-profiling="false"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592895    2569 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592899    2569 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592902    2569 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592904    2569 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592908    2569 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592911    2569 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592914    2569 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592917    2569 flags.go:64] FLAG: --enable-load-reader="false"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592920    2569 flags.go:64] FLAG: --enable-server="true"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592923    2569 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592927    2569 flags.go:64] FLAG: --event-burst="100"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592930    2569 flags.go:64] FLAG: --event-qps="50"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592933    2569 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592936    2569 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592939    2569 flags.go:64] FLAG: --eviction-hard=""
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592943    2569 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592946    2569 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592949    2569 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592952    2569 flags.go:64] FLAG: --eviction-soft=""
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592955    2569 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592957    2569 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 09 18:59:15.597442 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592961    2569 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592964    2569 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592967    2569 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592969    2569 flags.go:64] FLAG: --fail-swap-on="true"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592972    2569 flags.go:64] FLAG: --feature-gates=""
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592976    2569 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592978    2569 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592981    2569 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592984    2569 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592991    2569 flags.go:64] FLAG: --healthz-port="10248"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592994    2569 flags.go:64] FLAG: --help="false"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592996    2569 flags.go:64] FLAG: --hostname-override="ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.592999    2569 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593002    2569 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593005    2569 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593008    2569 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593012    2569 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593015    2569 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593017    2569 flags.go:64] FLAG: --image-service-endpoint=""
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593020    2569 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593025    2569 flags.go:64] FLAG: --kube-api-burst="100"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593037    2569 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593040    2569 flags.go:64] FLAG: --kube-api-qps="50"
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593043    2569 flags.go:64] FLAG: --kube-reserved=""
Jul 09 18:59:15.598096 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593046    2569 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593049    2569 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593052    2569 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593054    2569 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593057    2569 flags.go:64] FLAG: --lock-file=""
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593060    2569 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593063    2569 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593066    2569 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593071    2569 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593075    2569 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593078    2569 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593080    2569 flags.go:64] FLAG: --logging-format="text"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593083    2569 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593086    2569 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593089    2569 flags.go:64] FLAG: --manifest-url=""
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593092    2569 flags.go:64] FLAG: --manifest-url-header=""
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593101    2569 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593105    2569 flags.go:64] FLAG: --max-open-files="1000000"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593109    2569 flags.go:64] FLAG: --max-pods="110"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593112    2569 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593115    2569 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593117    2569 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593120    2569 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593123    2569 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593126    2569 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 09 18:59:15.598680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593129    2569 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593136    2569 flags.go:64] FLAG: --node-status-max-images="50"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593138    2569 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593143    2569 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593146    2569 flags.go:64] FLAG: --pod-cidr=""
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593148    2569 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593154    2569 flags.go:64] FLAG: --pod-manifest-path=""
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593157    2569 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593160    2569 flags.go:64] FLAG: --pods-per-core="0"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593162    2569 flags.go:64] FLAG: --port="10250"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593165    2569 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593168    2569 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d99cbf0601151b0d"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593171    2569 flags.go:64] FLAG: --qos-reserved=""
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593174    2569 flags.go:64] FLAG: --read-only-port="10255"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593177    2569 flags.go:64] FLAG: --register-node="true"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593180    2569 flags.go:64] FLAG: --register-with-taints=""
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593183    2569 flags.go:64] FLAG: --registry-burst="10"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593186    2569 flags.go:64] FLAG: --registry-qps="5"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593189    2569 flags.go:64] FLAG: --reserved-cpus=""
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593192    2569 flags.go:64] FLAG: --reserved-memory=""
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593195    2569 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593198    2569 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593201    2569 flags.go:64] FLAG: --rotate-certificates="false"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593204    2569 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593206    2569 flags.go:64] FLAG: --runonce="false"
Jul 09 18:59:15.599310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593210    2569 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593213    2569 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593216    2569 flags.go:64] FLAG: --seccomp-default="false"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593219    2569 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593222    2569 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593225    2569 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593228    2569 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593231    2569 flags.go:64] FLAG: --storage-driver-password="root"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593234    2569 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593236    2569 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593240    2569 flags.go:64] FLAG: --storage-driver-user="root"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593243    2569 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593246    2569 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593249    2569 flags.go:64] FLAG: --system-cgroups=""
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593252    2569 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593257    2569 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593260    2569 flags.go:64] FLAG: --tls-cert-file=""
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593262    2569 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593266    2569 flags.go:64] FLAG: --tls-min-version=""
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593269    2569 flags.go:64] FLAG: --tls-private-key-file=""
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593271    2569 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593274    2569 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593277    2569 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593280    2569 flags.go:64] FLAG: --v="2"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593284    2569 flags.go:64] FLAG: --version="false"
Jul 09 18:59:15.599950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593289    2569 flags.go:64] FLAG: --vmodule=""
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593292    2569 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.593296    2569 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593413    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593419    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593423    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593426    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593430    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593433    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593436    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593438    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593441    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593443    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593447    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593450    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593454    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593458    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593463    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:59:15.600585 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593465    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593468    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593471    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593474    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593476    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593479    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593481    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593484    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593486    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593489    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593491    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593494    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593496    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593499    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593501    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593505    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593507    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593509    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593512    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593514    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:59:15.601076 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593517    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593521    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593523    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593526    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593528    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593531    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593534    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593536    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593538    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593541    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593543    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593547    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593550    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593553    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593555    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593558    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593560    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593562    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593565    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593568    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:59:15.601550 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593570    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593573    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593575    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593577    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593580    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593583    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593585    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593587    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593591    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593593    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593595    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593598    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593601    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593604    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593607    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593609    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593612    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593614    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593617    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593619    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:59:15.602041 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593622    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593624    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593627    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593630    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593633    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593635    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593638    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593641    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593643    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593646    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593648    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593651    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593653    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593656    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593658    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593660    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593663    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593665    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593668    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593670    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:59:15.602517 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593672    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593676    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593678    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.593681    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.594192    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.601087    2569 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.601103    2569 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.601126    2569 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.601134    2569 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601193    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601199    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601202    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601205    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601207    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:59:15.603027 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601210    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601215    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601219    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601222    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601225    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601229    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601232    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601236    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601239    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601242    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601245    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601247    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601250    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601253    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601256    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601259    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601261    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601264    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601267    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:59:15.603375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601270    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601272    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601276    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601279    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601282    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601286    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601289    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601292    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601295    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601297    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601300    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601302    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601305    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601307    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601309    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601312    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601315    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601317    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601320    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:59:15.603838 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601322    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601325    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601328    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601331    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601333    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601335    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601338    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601341    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601343    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601346    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601349    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601351    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601353    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601356    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601359    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601362    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601364    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601367    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601370    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601373    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:59:15.604375 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601375    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601378    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601380    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601383    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601385    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601388    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601390    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601392    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601395    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601397    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601400    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601402    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601404    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601407    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601409    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601411    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601414    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601416    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601419    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601421    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:59:15.604860 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601423    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601426    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601429    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601431    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601434    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601436    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601438    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601442    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601444    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601447    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601449    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601452    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601455    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601457    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601460    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:59:15.605364 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601462    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.601467    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601598    2569 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601603    2569 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601606    2569 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601609    2569 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601612    2569 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601615    2569 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601617    2569 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601620    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601622    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601625    2569 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601627    2569 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601630    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 09 18:59:15.605725 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601633    2569 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601635    2569 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601637    2569 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601640    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601642    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601645    2569 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601648    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601650    2569 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601653    2569 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601655    2569 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601658    2569 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601660    2569 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601663    2569 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601665    2569 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601668    2569 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601670    2569 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601673    2569 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601675    2569 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601678    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601680    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 09 18:59:15.606075 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601682    2569 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601685    2569 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601687    2569 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601690    2569 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601692    2569 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601694    2569 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601697    2569 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601699    2569 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601703    2569 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601706    2569 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601709    2569 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601712    2569 feature_gate.go:328] unrecognized feature gate: Example2
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601715    2569 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601718    2569 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601721    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601723    2569 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601726    2569 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601729    2569 feature_gate.go:328] unrecognized feature gate: Example
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601731    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 09 18:59:15.606548 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601733    2569 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601736    2569 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601739    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601741    2569 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601743    2569 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601746    2569 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601749    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601751    2569 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601753    2569 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601757    2569 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601760    2569 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601763    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601766    2569 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601768    2569 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601771    2569 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601774    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601776    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601779    2569 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601782    2569 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601785    2569 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 09 18:59:15.607023 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601787    2569 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601790    2569 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601793    2569 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601796    2569 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601798    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601800    2569 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601803    2569 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601805    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601808    2569 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601811    2569 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601813    2569 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601816    2569 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601818    2569 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601820    2569 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601823    2569 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601825    2569 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601828    2569 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601830    2569 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601833    2569 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 09 18:59:15.607491 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601836    2569 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601838    2569 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601841    2569 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601843    2569 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601845    2569 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601848    2569 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601852    2569 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601855    2569 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:15.601857    2569 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.601862    2569 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.601982    2569 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.603945    2569 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.605194    2569 server.go:1019] "Starting client certificate rotation"
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.605301    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:59:15.608033 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.605333    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 09 18:59:15.608372 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.607527    2569 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:59:15.611262 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.611247    2569 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 09 18:59:15.621363 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.620772    2569 log.go:25] "Validated CRI v1 runtime API"
Jul 09 18:59:15.623934 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.623916    2569 log.go:25] "Validated CRI v1 image API"
Jul 09 18:59:15.624721 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.624706    2569 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 09 18:59:15.627009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.626992    2569 fs.go:135] Filesystem UUIDs: map[1676937e-d344-4f99-bdba-2537bb37739f:/dev/nvme0n1p3 2345b37f-98fe-436f-949f-10825c9cd514:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 09 18:59:15.627067 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.627009    2569 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 09 18:59:15.632993 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.632882    2569 manager.go:216] Machine: {Timestamp:2026-07-09 18:59:15.631002287 +0000 UTC m=+0.317247200 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3113828 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec24a20c0d736ddbe5ddd30b60298ed6 SystemUUID:ec24a20c-0d73-6ddb-e5dd-d30b60298ed6 BootID:12f94bc7-e693-42cb-a132-f5d4d9941260 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8f:03:f5:1d:83 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8f:03:f5:1d:83 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:de:83:21:e7:cc:39 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 09 18:59:15.632993 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.632986    2569 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 09 18:59:15.633117 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.633061    2569 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 09 18:59:15.633671 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.633656    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:59:15.634009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.633985    2569 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 09 18:59:15.634150 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.634013    2569 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-157.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 09 18:59:15.635449 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.635438    2569 topology_manager.go:138] "Creating topology manager with none policy"
Jul 09 18:59:15.635449 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.635450    2569 container_manager_linux.go:309] "Creating device plugin manager"
Jul 09 18:59:15.635512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.635457    2569 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:59:15.636213 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.636202    2569 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 09 18:59:15.637587 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.637577    2569 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 09 18:59:15.640223 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640212    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:59:15.640343 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640333    2569 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 09 18:59:15.640382 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640366    2569 kubelet.go:486] "Attempting to sync node with API server"
Jul 09 18:59:15.640417 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640383    2569 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 09 18:59:15.640417 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640394    2569 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 09 18:59:15.640417 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640404    2569 kubelet.go:398] "Adding apiserver pod source"
Jul 09 18:59:15.640417 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640412    2569 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 09 18:59:15.640771 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640755    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:59:15.640804 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.640781    2569 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 09 18:59:15.643050 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.643035    2569 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 09 18:59:15.644936 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.644922    2569 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 09 18:59:15.644988 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.644953    2569 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 09 18:59:15.645123 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645111    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 09 18:59:15.645157 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645133    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 09 18:59:15.645157 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645143    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 09 18:59:15.645157 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645151    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 09 18:59:15.645235 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645162    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 09 18:59:15.645235 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645169    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 09 18:59:15.645551 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645542    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 09 18:59:15.645579 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645555    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 09 18:59:15.645579 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645566    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 09 18:59:15.645579 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645572    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 09 18:59:15.645579 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645580    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 09 18:59:15.645680 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.645589    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 09 18:59:15.646473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.646463    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 09 18:59:15.646473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.646473    2569 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 09 18:59:15.649359 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.649328    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 09 18:59:15.649441 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.649337    2569 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-157.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 09 18:59:15.649441 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.649380    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 09 18:59:15.649788 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.649774    2569 server.go:1268] "Started kubelet"
Jul 09 18:59:15.649890 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.649846    2569 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 09 18:59:15.649943 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.649912    2569 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 09 18:59:15.650009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.649961    2569 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 09 18:59:15.650895 ip-10-0-128-157 systemd[1]: Started Kubernetes Kubelet.
Jul 09 18:59:15.651027 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.651011    2569 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 09 18:59:15.655581 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.655545    2569 server.go:310] "Adding debug handlers to kubelet server"
Jul 09 18:59:15.656538 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.656517    2569 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 09 18:59:15.657334 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.656472    2569 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-157.ec2.internal.18c0b42c7f85c542  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-157.ec2.internal,UID:ip-10-0-128-157.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-157.ec2.internal,},FirstTimestamp:2026-07-09 18:59:15.649750338 +0000 UTC m=+0.335995251,LastTimestamp:2026-07-09 18:59:15.649750338 +0000 UTC m=+0.335995251,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-157.ec2.internal,}"
Jul 09 18:59:15.658255 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.658235    2569 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 09 18:59:15.658255 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.658249    2569 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 09 18:59:15.658397 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.658315    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:15.659128 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659112    2569 factory.go:153] Registering CRI-O factory
Jul 09 18:59:15.659128 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659127    2569 factory.go:223] Registration of the crio container factory successfully
Jul 09 18:59:15.659263 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659178    2569 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 09 18:59:15.659263 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659187    2569 factory.go:55] Registering systemd factory
Jul 09 18:59:15.659263 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659195    2569 factory.go:223] Registration of the systemd container factory successfully
Jul 09 18:59:15.659263 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659214    2569 factory.go:103] Registering Raw factory
Jul 09 18:59:15.659263 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659227    2569 manager.go:1196] Started watching for new ooms in manager
Jul 09 18:59:15.659613 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659428    2569 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 09 18:59:15.659613 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659455    2569 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 09 18:59:15.659613 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659499    2569 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 09 18:59:15.659843 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659621    2569 reconstruct.go:97] "Volume reconstruction finished"
Jul 09 18:59:15.659843 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659630    2569 reconciler.go:29] "Reconciler: start to sync state"
Jul 09 18:59:15.659959 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.659920    2569 manager.go:349] Starting recovery of all containers
Jul 09 18:59:15.661800 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.661765    2569 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-157.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 09 18:59:15.661895 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.661806    2569 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 09 18:59:15.662480 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.662456    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fxrxn"
Jul 09 18:59:15.670133 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.669978    2569 manager.go:354] Recovery completed
Jul 09 18:59:15.670290 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.670276    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fxrxn"
Jul 09 18:59:15.674462 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.674449    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:59:15.676144 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.676129    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:59:15.676216 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.676162    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:59:15.676216 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.676175    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:59:15.676665 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.676650    2569 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 09 18:59:15.676665 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.676664    2569 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 09 18:59:15.676775 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.676680    2569 state_mem.go:36] "Initialized new in-memory state store"
Jul 09 18:59:15.676775 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.676697    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:59:15.677329 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.677314    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:59:15.677380 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.677341    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:59:15.677380 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.677353    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:59:15.677849 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.677828    2569 policy_none.go:49] "None policy: Start"
Jul 09 18:59:15.677993 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.677981    2569 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 09 18:59:15.678087 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.678077    2569 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 09 18:59:15.679444 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.679358    2569 policy_none.go:47] "Start"
Jul 09 18:59:15.716450 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.716428    2569 manager.go:337] "Starting Device Plugin manager"
Jul 09 18:59:15.716552 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.716464    2569 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 09 18:59:15.716552 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.716477    2569 server.go:87] "Starting device plugin registration server"
Jul 09 18:59:15.716742 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.716730    2569 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 09 18:59:15.716800 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.716745    2569 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 09 18:59:15.716887 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.716853    2569 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 09 18:59:15.716966 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.716954    2569 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 09 18:59:15.716966 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.716966    2569 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 09 18:59:15.717416 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.717398    2569 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 09 18:59:15.717495 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.717451    2569 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:15.766648 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.766616    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 09 18:59:15.769067 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.767941    2569 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 09 18:59:15.769067 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.767983    2569 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 09 18:59:15.769067 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.768008    2569 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 09 18:59:15.769067 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.768047    2569 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 09 18:59:15.770043 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.770025    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:59:15.817182 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.817152    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:59:15.818233 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.818216    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:59:15.818310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.818251    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:59:15.818310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.818264    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:59:15.818310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.818287    2569 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.823549 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.823535    2569 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.823601 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.823557    2569 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-157.ec2.internal\": node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:15.838999 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.838952    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:15.868936 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.868912    2569 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal"]
Jul 09 18:59:15.868986 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.868974    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:59:15.869776 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.869762    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:59:15.869836 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.869788    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:59:15.869836 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.869798    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:59:15.871307 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871295    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:59:15.871425 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871410    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.871472 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871436    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:59:15.871950 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871929    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:59:15.872015 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871954    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:59:15.872015 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871978    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:59:15.872015 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871990    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:59:15.872112 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.871957    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:59:15.872112 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.872052    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:59:15.873490 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.873475    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.873572 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.873502    2569 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 09 18:59:15.874160 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.874146    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientMemory"
Jul 09 18:59:15.874214 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.874172    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasNoDiskPressure"
Jul 09 18:59:15.874214 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.874183    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeHasSufficientPID"
Jul 09 18:59:15.901441 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.901422    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-157.ec2.internal\" not found" node="ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.905794 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.905781    2569 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-157.ec2.internal\" not found" node="ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.940008 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:15.939990    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:15.961165 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.961144    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dcf208c2c2ddf8d2ac591147d73def93-config\") pod \"kube-apiserver-proxy-ip-10-0-128-157.ec2.internal\" (UID: \"dcf208c2c2ddf8d2ac591147d73def93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.961229 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.961178    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31ce84329abd6603f35db6b08a63b7f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal\" (UID: \"31ce84329abd6603f35db6b08a63b7f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:15.961229 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:15.961196    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31ce84329abd6603f35db6b08a63b7f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal\" (UID: \"31ce84329abd6603f35db6b08a63b7f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.040380 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.040351    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.061730 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.061708    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31ce84329abd6603f35db6b08a63b7f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal\" (UID: \"31ce84329abd6603f35db6b08a63b7f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.061798 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.061733    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dcf208c2c2ddf8d2ac591147d73def93-config\") pod \"kube-apiserver-proxy-ip-10-0-128-157.ec2.internal\" (UID: \"dcf208c2c2ddf8d2ac591147d73def93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.061798 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.061754    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31ce84329abd6603f35db6b08a63b7f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal\" (UID: \"31ce84329abd6603f35db6b08a63b7f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.061893 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.061803    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/31ce84329abd6603f35db6b08a63b7f4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal\" (UID: \"31ce84329abd6603f35db6b08a63b7f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.061893 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.061817    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/31ce84329abd6603f35db6b08a63b7f4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal\" (UID: \"31ce84329abd6603f35db6b08a63b7f4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.061893 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.061821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dcf208c2c2ddf8d2ac591147d73def93-config\") pod \"kube-apiserver-proxy-ip-10-0-128-157.ec2.internal\" (UID: \"dcf208c2c2ddf8d2ac591147d73def93\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.141039 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.140967    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.203540 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.203501    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.208578 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.208563    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.241141 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.241111    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.341644 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.341611    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.442239 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.442170    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.542830 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.542801    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.604291 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.604268    2569 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 09 18:59:16.604767 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.604398    2569 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 09 18:59:16.643724 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.643701    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.673696 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.673654    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-08 18:54:17 +0000 UTC" deadline="2028-01-05 09:49:27.162913351 +0000 UTC"
Jul 09 18:59:16.673696 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.673696    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13070h50m10.489221761s"
Jul 09 18:59:16.744651 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:16.744587    2569 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-157.ec2.internal\" not found"
Jul 09 18:59:16.761643 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.761621    2569 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:59:16.763350 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:16.763327    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcf208c2c2ddf8d2ac591147d73def93.slice/crio-e8845fec6e7f4d49132b631b8c34304502da175c8f026cec7b2144908d2043d7 WatchSource:0}: Error finding container e8845fec6e7f4d49132b631b8c34304502da175c8f026cec7b2144908d2043d7: Status 404 returned error can't find the container with id e8845fec6e7f4d49132b631b8c34304502da175c8f026cec7b2144908d2043d7
Jul 09 18:59:16.763518 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:16.763503    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31ce84329abd6603f35db6b08a63b7f4.slice/crio-aae443c71405dfbd955655f29835fcad7d62c763e2e4b115aa47c31eaf7dcbee WatchSource:0}: Error finding container aae443c71405dfbd955655f29835fcad7d62c763e2e4b115aa47c31eaf7dcbee: Status 404 returned error can't find the container with id aae443c71405dfbd955655f29835fcad7d62c763e2e4b115aa47c31eaf7dcbee
Jul 09 18:59:16.768154 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.768138    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 18:59:16.771945 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.771898    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal" event={"ID":"31ce84329abd6603f35db6b08a63b7f4","Type":"ContainerStarted","Data":"aae443c71405dfbd955655f29835fcad7d62c763e2e4b115aa47c31eaf7dcbee"}
Jul 09 18:59:16.772805 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.772788    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal" event={"ID":"dcf208c2c2ddf8d2ac591147d73def93","Type":"ContainerStarted","Data":"e8845fec6e7f4d49132b631b8c34304502da175c8f026cec7b2144908d2043d7"}
Jul 09 18:59:16.818105 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.818077    2569 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:59:16.858980 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.858958    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.859694 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.859680    2569 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:59:16.869395 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.869374    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:59:16.870330 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.870318    2569 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal"
Jul 09 18:59:16.881592 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:16.881576    2569 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 09 18:59:19.595535 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.595461    2569 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 09 18:59:19.744727 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.744611    2569 apiserver.go:52] "Watching apiserver"
Jul 09 18:59:19.756765 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.756738    2569 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 09 18:59:19.757912 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.757886    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-6tmrs","openshift-network-diagnostics/network-check-target-2r4wh","kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal","openshift-multus/network-metrics-daemon-srwqt","openshift-network-operator/iptables-alerter-nw8ld","openshift-ovn-kubernetes/ovnkube-node-drt6n","kube-system/global-pull-secret-syncer-9tnc2","kube-system/konnectivity-agent-x48nr","openshift-cluster-node-tuning-operator/tuned-grm29","openshift-dns/node-resolver-f5bwn","openshift-image-registry/node-ca-gftdt","openshift-multus/multus-6z8dd"]
Jul 09 18:59:19.759466 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.759447    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.760535 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.760516    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:19.760623 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.760586    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:19.762377 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.761525    2569 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 09 18:59:19.762377 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.761779    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.762522 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.762510    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 09 18:59:19.762574 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.762520    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 09 18:59:19.762742 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.762731    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-w7qxw\""
Jul 09 18:59:19.762891 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.762864    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 09 18:59:19.763003 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.762989    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 09 18:59:19.763115 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.763084    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 09 18:59:19.763656 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.763348    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.764294 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.764276    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 09 18:59:19.764647 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.764626    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qcjkn\""
Jul 09 18:59:19.765408 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.765392    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.766243 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.766225    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 09 18:59:19.766409 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.766390    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2vtnp\""
Jul 09 18:59:19.766728 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.766555    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 09 18:59:19.766728 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.766615    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 09 18:59:19.766884 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.766816    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.767640 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.767587    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.767841 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.767824    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 09 18:59:19.768146 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.768130    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 09 18:59:19.768226 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.768214    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-s9mqf\""
Jul 09 18:59:19.768774 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.768435    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 09 18:59:19.769411 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.769390    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 09 18:59:19.769508 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.769483    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-lkwqp\""
Jul 09 18:59:19.769754 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.769726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.769834 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.769764    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:59:19.771735 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.770605    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:59:19.771735 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.770738    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wstz2\""
Jul 09 18:59:19.771735 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.770927    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 09 18:59:19.771735 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.771154    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 09 18:59:19.771735 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.771721    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.771969 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.771779    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772376    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772399    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772548    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dzkpn\""
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772403    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xphvk\" (UniqueName: \"kubernetes.io/projected/a46e7512-0450-4b03-8ff0-a9e93d82166f-kube-api-access-xphvk\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772669    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-cni-multus\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-kubelet\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772549    2569 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772625    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772744    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772834    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cnibin\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.772949 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772902    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-cni-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772987    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.772980    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-etc-kubernetes\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773029    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-socket-dir-parent\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-cni-bin\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773074    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773097    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-netns\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773142    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-multus-certs\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773163    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-socket-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773186    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-sys-fs\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773222    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-os-release\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773290    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vx4l2\" (UniqueName: \"kubernetes.io/projected/feeda688-dbc8-4a69-8c1c-e7487aad15ba-kube-api-access-vx4l2\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-system-cni-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773390    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zqx5\" (UniqueName: \"kubernetes.io/projected/20730733-2d51-4667-acf3-374120aa43c5-kube-api-access-6zqx5\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773423    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.773627 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773453    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-cnibin\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20730733-2d51-4667-acf3-374120aa43c5-cni-binary-copy\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773494    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-system-cni-dir\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773521    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-conf-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773547    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-device-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773571    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773600    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-os-release\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773614    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-hostroot\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773628    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20730733-2d51-4667-acf3-374120aa43c5-multus-daemon-config\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773642    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-k8s-cni-cncf-io\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.773657    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-registration-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.774066    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.774103    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 09 18:59:19.774473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.774302    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 09 18:59:19.775072 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.774943    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:19.775072 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.774985    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:19.775598 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.775347    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 09 18:59:19.775598 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.775507    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-q6cpk\""
Jul 09 18:59:19.775763 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.775737    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.777448 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.777428    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 09 18:59:19.778364 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.778346    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 09 18:59:19.778455 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.778402    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 09 18:59:19.778455 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.778436    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-hsvnj\""
Jul 09 18:59:19.795606 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.795581    2569 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2jq9f"
Jul 09 18:59:19.804064 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.804040    2569 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2jq9f"
Jul 09 18:59:19.863199 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.863138    2569 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 09 18:59:19.873900 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.873877    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-var-lib-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.874009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.873904    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08bf4c59-eb78-4b76-ac07-eefab77990bd-agent-certs\") pod \"konnectivity-agent-x48nr\" (UID: \"08bf4c59-eb78-4b76-ac07-eefab77990bd\") " pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:19.874009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.873924    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.874009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.873938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-etc-kubernetes\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.874009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.873961    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsxg2\" (UniqueName: \"kubernetes.io/projected/211d2466-c017-42ae-a8f7-8321c52f6c66-kube-api-access-gsxg2\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.874009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.873976    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-etc-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.874009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.873989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-run-ovn-kubernetes\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.874009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08bf4c59-eb78-4b76-ac07-eefab77990bd-konnectivity-ca\") pod \"konnectivity-agent-x48nr\" (UID: \"08bf4c59-eb78-4b76-ac07-eefab77990bd\") " pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874016    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-dbus\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874039    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrgw5\" (UniqueName: \"kubernetes.io/projected/503853d6-c3ef-4994-9151-a4517a4c0abb-kube-api-access-mrgw5\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874054    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-run\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-slash\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874091    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-systemd\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874112    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-sys\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-cni-bin\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874211    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874235    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysconfig\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874268    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.874313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874293    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-socket-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874348    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-sys-fs\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-os-release\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874384    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vx4l2\" (UniqueName: \"kubernetes.io/projected/feeda688-dbc8-4a69-8c1c-e7487aad15ba-kube-api-access-vx4l2\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874400    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874452    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-kubernetes\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-ovn\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874495    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874520    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-cnibin\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874541    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20730733-2d51-4667-acf3-374120aa43c5-cni-binary-copy\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874577    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-lib-modules\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874597    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-968w8\" (UniqueName: \"kubernetes.io/projected/060e1b6b-c1bd-4c2f-a967-2635e32c69de-kube-api-access-968w8\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/211d2466-c017-42ae-a8f7-8321c52f6c66-host-slash\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874633    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-system-cni-dir\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874648    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-conf-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.874705 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874676    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-host\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874717    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-kubelet\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874713    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874794    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-systemd-units\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874812    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovnkube-config\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874861    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20730733-2d51-4667-acf3-374120aa43c5-multus-daemon-config\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874876    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysctl-conf\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874899    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-var-lib-kubelet\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-k8s-cni-cncf-io\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874938    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xphvk\" (UniqueName: \"kubernetes.io/projected/a46e7512-0450-4b03-8ff0-a9e93d82166f-kube-api-access-xphvk\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874958    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-kubelet\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.874990    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9524d05b-67d4-4944-9403-1d5d9a9e7872-serviceca\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875004    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-cni-netd\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875049    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-etc-kubernetes\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875095    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-cnibin\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.875295 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875216    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-cni-bin\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875364    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875453    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-socket-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875496    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-sys-fs\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875536    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/20730733-2d51-4667-acf3-374120aa43c5-cni-binary-copy\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875547    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-os-release\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875585    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-system-cni-dir\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875621    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb527e7-82f3-4a04-ac64-789566abc7ac-tmp-dir\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.875844 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-conf-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875890    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cnibin\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875955    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-cni-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875980    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/20730733-2d51-4667-acf3-374120aa43c5-multus-daemon-config\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.875989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-run-netns\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-kubelet-config\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876042    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-kubelet\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876062    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-socket-dir-parent\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876098    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9524d05b-67d4-4944-9403-1d5d9a9e7872-host\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-systemd\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876149    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-tuned\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-socket-dir-parent\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876170    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/060e1b6b-c1bd-4c2f-a967-2635e32c69de-tmp\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876189    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-k8s-cni-cncf-io\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876128    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-multus-cni-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876195    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hsjx\" (UniqueName: \"kubernetes.io/projected/5bb527e7-82f3-4a04-ac64-789566abc7ac-kube-api-access-8hsjx\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876232    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876263    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-netns\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876289    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-multus-certs\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876316    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876336    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-node-log\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876354    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-log-socket\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876373    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysctl-d\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876410    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-system-cni-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876438    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zqx5\" (UniqueName: \"kubernetes.io/projected/20730733-2d51-4667-acf3-374120aa43c5-kube-api-access-6zqx5\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876454    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-system-cni-dir\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876477    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-env-overrides\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876484    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876505    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovnkube-script-lib\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.876668 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876507    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cnibin\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876533    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hltb8\" (UniqueName: \"kubernetes.io/projected/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-kube-api-access-hltb8\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876559    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bb527e7-82f3-4a04-ac64-789566abc7ac-hosts-file\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876592    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-etc-selinux\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876631    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876661    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x89bq\" (UniqueName: \"kubernetes.io/projected/9524d05b-67d4-4944-9403-1d5d9a9e7872-kube-api-access-x89bq\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876710    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/211d2466-c017-42ae-a8f7-8321c52f6c66-iptables-alerter-script\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876740    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovn-node-metrics-cert\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876749    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-netns\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876784    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-run-multus-certs\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876829    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-device-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876869    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876924    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-device-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.876965    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-os-release\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-hostroot\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877138    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-hostroot\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877173    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-cni-bin\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.877478 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-registration-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.878109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-os-release\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.878109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877279    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.878109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877280    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-cni-multus\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.878109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877300    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a46e7512-0450-4b03-8ff0-a9e93d82166f-registration-dir\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.878109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/20730733-2d51-4667-acf3-374120aa43c5-host-var-lib-cni-multus\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.878109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877428    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feeda688-dbc8-4a69-8c1c-e7487aad15ba-cni-binary-copy\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.878109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.877434    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-modprobe-d\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.881304 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.881280    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:59:19.881402 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.881311    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:59:19.881402 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.881324    2569 projected.go:196] Error preparing data for projected volume kube-api-access-w888x for pod openshift-network-diagnostics/network-check-target-2r4wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:19.881402 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.881387    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x podName:63e67d89-c759-424a-9e4f-0c598af63919 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:20.381359479 +0000 UTC m=+2.964791654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-w888x" (UniqueName: "kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x") pod "network-check-target-2r4wh" (UID: "63e67d89-c759-424a-9e4f-0c598af63919") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:19.881565 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.881538    2569 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 09 18:59:19.884842 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.884821    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vx4l2\" (UniqueName: \"kubernetes.io/projected/feeda688-dbc8-4a69-8c1c-e7487aad15ba-kube-api-access-vx4l2\") pod \"multus-additional-cni-plugins-6tmrs\" (UID: \"feeda688-dbc8-4a69-8c1c-e7487aad15ba\") " pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:19.889036 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.889016    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xphvk\" (UniqueName: \"kubernetes.io/projected/a46e7512-0450-4b03-8ff0-a9e93d82166f-kube-api-access-xphvk\") pod \"aws-ebs-csi-driver-node-p4j77\" (UID: \"a46e7512-0450-4b03-8ff0-a9e93d82166f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:19.891649 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.891630    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zqx5\" (UniqueName: \"kubernetes.io/projected/20730733-2d51-4667-acf3-374120aa43c5-kube-api-access-6zqx5\") pod \"multus-6z8dd\" (UID: \"20730733-2d51-4667-acf3-374120aa43c5\") " pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:19.978246 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978217    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9524d05b-67d4-4944-9403-1d5d9a9e7872-host\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.978246 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978252    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-systemd\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978275    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-tuned\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978296    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/060e1b6b-c1bd-4c2f-a967-2635e32c69de-tmp\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978316    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hsjx\" (UniqueName: \"kubernetes.io/projected/5bb527e7-82f3-4a04-ac64-789566abc7ac-kube-api-access-8hsjx\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9524d05b-67d4-4944-9403-1d5d9a9e7872-host\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978374    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-node-log\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978393    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-log-socket\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978341    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-systemd\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978412    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysctl-d\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978452    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-env-overrides\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978467    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-node-log\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.978487 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978478    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovnkube-script-lib\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978494    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978513    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hltb8\" (UniqueName: \"kubernetes.io/projected/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-kube-api-access-hltb8\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978539    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bb527e7-82f3-4a04-ac64-789566abc7ac-hosts-file\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978560    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-log-socket\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978568    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x89bq\" (UniqueName: \"kubernetes.io/projected/9524d05b-67d4-4944-9403-1d5d9a9e7872-kube-api-access-x89bq\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978594    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/211d2466-c017-42ae-a8f7-8321c52f6c66-iptables-alerter-script\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovn-node-metrics-cert\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978643    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-cni-bin\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978656    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysctl-d\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978677    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5bb527e7-82f3-4a04-ac64-789566abc7ac-hosts-file\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978714    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-modprobe-d\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978741    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-var-lib-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978766    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08bf4c59-eb78-4b76-ac07-eefab77990bd-agent-certs\") pod \"konnectivity-agent-x48nr\" (UID: \"08bf4c59-eb78-4b76-ac07-eefab77990bd\") " pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978794    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsxg2\" (UniqueName: \"kubernetes.io/projected/211d2466-c017-42ae-a8f7-8321c52f6c66-kube-api-access-gsxg2\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978822    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-etc-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-run-ovn-kubernetes\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978871    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08bf4c59-eb78-4b76-ac07-eefab77990bd-konnectivity-ca\") pod \"konnectivity-agent-x48nr\" (UID: \"08bf4c59-eb78-4b76-ac07-eefab77990bd\") " pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978895    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-dbus\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrgw5\" (UniqueName: \"kubernetes.io/projected/503853d6-c3ef-4994-9151-a4517a4c0abb-kube-api-access-mrgw5\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978932    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-var-lib-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978944    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-run\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.978984    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-slash\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.979004    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-run\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.979026    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-slash\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.979865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.979124    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovnkube-script-lib\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.982812 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.982785    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-env-overrides\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.979539    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-cni-bin\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.980169    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/211d2466-c017-42ae-a8f7-8321c52f6c66-iptables-alerter-script\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.980829    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-modprobe-d\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.980914    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-systemd\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.980925    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-run-ovn-kubernetes\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.981020    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-etc-openvswitch\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.980969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-systemd\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.982418    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/08bf4c59-eb78-4b76-ac07-eefab77990bd-konnectivity-ca\") pod \"konnectivity-agent-x48nr\" (UID: \"08bf4c59-eb78-4b76-ac07-eefab77990bd\") " pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.981569    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-dbus\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.982953 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.982946    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-sys\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.982995    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-sys\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.982946    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-tuned\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysconfig\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983117    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysconfig\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983188    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983259    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-kubernetes\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983286    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-ovn\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983355    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983383    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-lib-modules\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983410    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-968w8\" (UniqueName: \"kubernetes.io/projected/060e1b6b-c1bd-4c2f-a967-2635e32c69de-kube-api-access-968w8\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/211d2466-c017-42ae-a8f7-8321c52f6c66-host-slash\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.983502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983469    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-host\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983644    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-kubelet\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983669    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-systemd-units\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovnkube-config\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983774    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysctl-conf\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983851    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-var-lib-kubelet\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983883    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9524d05b-67d4-4944-9403-1d5d9a9e7872-serviceca\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983935    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-cni-netd\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983937    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/211d2466-c017-42ae-a8f7-8321c52f6c66-host-slash\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.983996    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb527e7-82f3-4a04-ac64-789566abc7ac-tmp-dir\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984000    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984034    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-run-netns\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984085    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-kubelet-config\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.984103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984094    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-kubernetes\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.984764 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-cni-netd\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984764 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984240    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-var-lib-kubelet\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.984764 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984569    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5bb527e7-82f3-4a04-ac64-789566abc7ac-tmp-dir\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.984764 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984648    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-run-netns\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.984764 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984711    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9524d05b-67d4-4944-9403-1d5d9a9e7872-serviceca\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.985047 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984765    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-kubelet-config\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:19.985047 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984839    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-host\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.985047 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984868    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-etc-sysctl-conf\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.985047 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984902    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-host-kubelet\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.985047 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984963    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-run-ovn\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.985047 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.984983    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-systemd-units\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.985269 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.985078    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:19.986109 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.985323    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret podName:ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:20.485297937 +0000 UTC m=+3.068730118 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret") pod "global-pull-secret-syncer-9tnc2" (UID: "ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:19.986109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.985342    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/060e1b6b-c1bd-4c2f-a967-2635e32c69de-tmp\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.986109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.985636    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/060e1b6b-c1bd-4c2f-a967-2635e32c69de-lib-modules\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:19.986109 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.985752    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovnkube-config\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.986109 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.985848    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:19.986109 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:19.985932    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs podName:503853d6-c3ef-4994-9151-a4517a4c0abb nodeName:}" failed. No retries permitted until 2026-07-09 18:59:20.485909234 +0000 UTC m=+3.069341406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs") pod "network-metrics-daemon-srwqt" (UID: "503853d6-c3ef-4994-9151-a4517a4c0abb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:19.986669 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.986486    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-ovn-node-metrics-cert\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.987915 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.987895    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x89bq\" (UniqueName: \"kubernetes.io/projected/9524d05b-67d4-4944-9403-1d5d9a9e7872-kube-api-access-x89bq\") pod \"node-ca-gftdt\" (UID: \"9524d05b-67d4-4944-9403-1d5d9a9e7872\") " pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:19.988416 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.988381    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/08bf4c59-eb78-4b76-ac07-eefab77990bd-agent-certs\") pod \"konnectivity-agent-x48nr\" (UID: \"08bf4c59-eb78-4b76-ac07-eefab77990bd\") " pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:19.988416 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.988403    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hltb8\" (UniqueName: \"kubernetes.io/projected/2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73-kube-api-access-hltb8\") pod \"ovnkube-node-drt6n\" (UID: \"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73\") " pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:19.988724 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.988704    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hsjx\" (UniqueName: \"kubernetes.io/projected/5bb527e7-82f3-4a04-ac64-789566abc7ac-kube-api-access-8hsjx\") pod \"node-resolver-f5bwn\" (UID: \"5bb527e7-82f3-4a04-ac64-789566abc7ac\") " pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:19.992932 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.992908    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsxg2\" (UniqueName: \"kubernetes.io/projected/211d2466-c017-42ae-a8f7-8321c52f6c66-kube-api-access-gsxg2\") pod \"iptables-alerter-nw8ld\" (UID: \"211d2466-c017-42ae-a8f7-8321c52f6c66\") " pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:19.993615 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.993596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrgw5\" (UniqueName: \"kubernetes.io/projected/503853d6-c3ef-4994-9151-a4517a4c0abb-kube-api-access-mrgw5\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:19.993741 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:19.993725    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-968w8\" (UniqueName: \"kubernetes.io/projected/060e1b6b-c1bd-4c2f-a967-2635e32c69de-kube-api-access-968w8\") pod \"tuned-grm29\" (UID: \"060e1b6b-c1bd-4c2f-a967-2635e32c69de\") " pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:20.072470 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.072442    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6tmrs"
Jul 09 18:59:20.080376 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.080357    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-6z8dd"
Jul 09 18:59:20.096310 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.096286    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77"
Jul 09 18:59:20.100829 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.100807    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gftdt"
Jul 09 18:59:20.110205 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.110185    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-grm29"
Jul 09 18:59:20.117859 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.117808    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nw8ld"
Jul 09 18:59:20.126521 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.126502    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:20.132244 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.132227    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:20.136856 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.136840    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-f5bwn"
Jul 09 18:59:20.388050 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.387977    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:20.388192 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.388139    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:59:20.388192 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.388158    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:59:20.388192 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.388167    2569 projected.go:196] Error preparing data for projected volume kube-api-access-w888x for pod openshift-network-diagnostics/network-check-target-2r4wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:20.388359 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.388229    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x podName:63e67d89-c759-424a-9e4f-0c598af63919 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:21.388207462 +0000 UTC m=+3.971639807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-w888x" (UniqueName: "kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x") pod "network-check-target-2r4wh" (UID: "63e67d89-c759-424a-9e4f-0c598af63919") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:20.489318 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.489290    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:20.489318 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.489323    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:20.489524 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.489432    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:20.489524 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.489443    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:20.489524 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.489481    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs podName:503853d6-c3ef-4994-9151-a4517a4c0abb nodeName:}" failed. No retries permitted until 2026-07-09 18:59:21.489467407 +0000 UTC m=+4.072899564 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs") pod "network-metrics-daemon-srwqt" (UID: "503853d6-c3ef-4994-9151-a4517a4c0abb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:20.489524 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:20.489495    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret podName:ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:21.48948866 +0000 UTC m=+4.072920818 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret") pod "global-pull-secret-syncer-9tnc2" (UID: "ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:20.765123 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:20.765095    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeeda688_dbc8_4a69_8c1c_e7487aad15ba.slice/crio-5ef09a0ec90d45325f37dd48bf4308af4f11f50e9ede23d43a450eba78fb7ce3 WatchSource:0}: Error finding container 5ef09a0ec90d45325f37dd48bf4308af4f11f50e9ede23d43a450eba78fb7ce3: Status 404 returned error can't find the container with id 5ef09a0ec90d45325f37dd48bf4308af4f11f50e9ede23d43a450eba78fb7ce3
Jul 09 18:59:20.770824 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:20.770634    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cfad3f0_1b7d_4ce4_8555_4b283f3f2f73.slice/crio-472493d67198ad89d70f599a5d9c8d85efab1643bc24b44e4524deadae212fbe WatchSource:0}: Error finding container 472493d67198ad89d70f599a5d9c8d85efab1643bc24b44e4524deadae212fbe: Status 404 returned error can't find the container with id 472493d67198ad89d70f599a5d9c8d85efab1643bc24b44e4524deadae212fbe
Jul 09 18:59:20.773944 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:20.773127    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9524d05b_67d4_4944_9403_1d5d9a9e7872.slice/crio-a73353c11be90de92cd0e760e1fcb402a7dce1bfcfc74ddab526766b47904f46 WatchSource:0}: Error finding container a73353c11be90de92cd0e760e1fcb402a7dce1bfcfc74ddab526766b47904f46: Status 404 returned error can't find the container with id a73353c11be90de92cd0e760e1fcb402a7dce1bfcfc74ddab526766b47904f46
Jul 09 18:59:20.774757 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:20.774734    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod060e1b6b_c1bd_4c2f_a967_2635e32c69de.slice/crio-2774a5d82c0a9b18e167555dbecefe5c6d86a6ae5d1d99dc138eb1647decf9bf WatchSource:0}: Error finding container 2774a5d82c0a9b18e167555dbecefe5c6d86a6ae5d1d99dc138eb1647decf9bf: Status 404 returned error can't find the container with id 2774a5d82c0a9b18e167555dbecefe5c6d86a6ae5d1d99dc138eb1647decf9bf
Jul 09 18:59:20.778311 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:20.778278    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08bf4c59_eb78_4b76_ac07_eefab77990bd.slice/crio-e4f0063d6edaad947df0da4dc16aa714f68161972dce3c75b54723bfbf594edb WatchSource:0}: Error finding container e4f0063d6edaad947df0da4dc16aa714f68161972dce3c75b54723bfbf594edb: Status 404 returned error can't find the container with id e4f0063d6edaad947df0da4dc16aa714f68161972dce3c75b54723bfbf594edb
Jul 09 18:59:20.779608 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:20.779210    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda46e7512_0450_4b03_8ff0_a9e93d82166f.slice/crio-5cb53dc88193471c3fe322ba9f70cbffe51b06fadb3d66ead0b3e0d2163b4e7f WatchSource:0}: Error finding container 5cb53dc88193471c3fe322ba9f70cbffe51b06fadb3d66ead0b3e0d2163b4e7f: Status 404 returned error can't find the container with id 5cb53dc88193471c3fe322ba9f70cbffe51b06fadb3d66ead0b3e0d2163b4e7f
Jul 09 18:59:20.807011 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.806862    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:54:19 +0000 UTC" deadline="2028-01-21 10:28:11.36508185 +0000 UTC"
Jul 09 18:59:20.807011 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.807010    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13455h28m50.558076734s"
Jul 09 18:59:20.881738 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.881710    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gftdt" event={"ID":"9524d05b-67d4-4944-9403-1d5d9a9e7872","Type":"ContainerStarted","Data":"a73353c11be90de92cd0e760e1fcb402a7dce1bfcfc74ddab526766b47904f46"}
Jul 09 18:59:20.883437 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.883414    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6z8dd" event={"ID":"20730733-2d51-4667-acf3-374120aa43c5","Type":"ContainerStarted","Data":"7e325cd1b1f15f81426ab3b7fef0e9fe767d140d80aafc85001b5a2bbe1e8956"}
Jul 09 18:59:20.885017 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.884997    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerStarted","Data":"5ef09a0ec90d45325f37dd48bf4308af4f11f50e9ede23d43a450eba78fb7ce3"}
Jul 09 18:59:20.886699 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.886648    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nw8ld" event={"ID":"211d2466-c017-42ae-a8f7-8321c52f6c66","Type":"ContainerStarted","Data":"1c014bc47892e2f3471bd0486bdc085c66adf212f87241432d6f758271973991"}
Jul 09 18:59:20.891174 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.891153    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"472493d67198ad89d70f599a5d9c8d85efab1643bc24b44e4524deadae212fbe"}
Jul 09 18:59:20.892386 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.892366    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f5bwn" event={"ID":"5bb527e7-82f3-4a04-ac64-789566abc7ac","Type":"ContainerStarted","Data":"945b4547b062bbc7c51370a0866a09b67c3b47259a2f5a88e993187be6d05e33"}
Jul 09 18:59:20.894858 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.894833    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal" event={"ID":"dcf208c2c2ddf8d2ac591147d73def93","Type":"ContainerStarted","Data":"fee9d186fc568a06fb26be65adc21f19e0b2bbea767bcd2a64689fbb11625d66"}
Jul 09 18:59:20.896531 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.896511    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77" event={"ID":"a46e7512-0450-4b03-8ff0-a9e93d82166f","Type":"ContainerStarted","Data":"5cb53dc88193471c3fe322ba9f70cbffe51b06fadb3d66ead0b3e0d2163b4e7f"}
Jul 09 18:59:20.898331 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.898311    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x48nr" event={"ID":"08bf4c59-eb78-4b76-ac07-eefab77990bd","Type":"ContainerStarted","Data":"e4f0063d6edaad947df0da4dc16aa714f68161972dce3c75b54723bfbf594edb"}
Jul 09 18:59:20.899346 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:20.899326    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-grm29" event={"ID":"060e1b6b-c1bd-4c2f-a967-2635e32c69de","Type":"ContainerStarted","Data":"2774a5d82c0a9b18e167555dbecefe5c6d86a6ae5d1d99dc138eb1647decf9bf"}
Jul 09 18:59:21.396574 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.396498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:21.396769 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.396646    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:59:21.396769 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.396662    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:59:21.396769 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.396671    2569 projected.go:196] Error preparing data for projected volume kube-api-access-w888x for pod openshift-network-diagnostics/network-check-target-2r4wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:21.396769 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.396739    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x podName:63e67d89-c759-424a-9e4f-0c598af63919 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:23.396721485 +0000 UTC m=+5.980153661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-w888x" (UniqueName: "kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x") pod "network-check-target-2r4wh" (UID: "63e67d89-c759-424a-9e4f-0c598af63919") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:21.497569 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.497532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:21.497779 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.497589    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:21.497779 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.497741    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:21.497885 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.497802    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs podName:503853d6-c3ef-4994-9151-a4517a4c0abb nodeName:}" failed. No retries permitted until 2026-07-09 18:59:23.49778384 +0000 UTC m=+6.081216014 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs") pod "network-metrics-daemon-srwqt" (UID: "503853d6-c3ef-4994-9151-a4517a4c0abb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:21.498208 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.498191    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:21.498277 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.498245    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret podName:ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:23.498230919 +0000 UTC m=+6.081663082 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret") pod "global-pull-secret-syncer-9tnc2" (UID: "ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:21.807208 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.807119    2569 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-08 18:54:19 +0000 UTC" deadline="2028-04-24 06:03:08.79760137 +0000 UTC"
Jul 09 18:59:21.807208 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.807160    2569 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15707h3m46.990446568s"
Jul 09 18:59:21.877162 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.876278    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:21.877162 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.876427    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:21.877162 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.876844    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:21.877162 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.876963    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:21.877162 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.877027    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:21.877162 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:21.877096    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:21.919250 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.918229    2569 generic.go:356] "Generic (PLEG): container finished" podID="31ce84329abd6603f35db6b08a63b7f4" containerID="8542bd14307b68ef662b2a535d7a544dfb30c08031f03f436245a2b0d726dae8" exitCode=0
Jul 09 18:59:21.919250 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.918928    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal" event={"ID":"31ce84329abd6603f35db6b08a63b7f4","Type":"ContainerDied","Data":"8542bd14307b68ef662b2a535d7a544dfb30c08031f03f436245a2b0d726dae8"}
Jul 09 18:59:21.941206 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:21.941154    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-157.ec2.internal" podStartSLOduration=3.941136771 podStartE2EDuration="3.941136771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:20.912739777 +0000 UTC m=+3.496171954" watchObservedRunningTime="2026-07-09 18:59:21.941136771 +0000 UTC m=+4.524568952"
Jul 09 18:59:22.929923 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:22.929813    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal" event={"ID":"31ce84329abd6603f35db6b08a63b7f4","Type":"ContainerStarted","Data":"bea6dd397e47cc918a10feaaa29d399a8157dceeec492484e39c243005d037d4"}
Jul 09 18:59:22.946769 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:22.946715    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-157.ec2.internal" podStartSLOduration=4.946675765 podStartE2EDuration="4.946675765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:22.945935252 +0000 UTC m=+5.529367433" watchObservedRunningTime="2026-07-09 18:59:22.946675765 +0000 UTC m=+5.530107947"
Jul 09 18:59:23.427139 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:23.426484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:23.427139 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.426647    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:59:23.427139 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.426666    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:59:23.427139 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.426694    2569 projected.go:196] Error preparing data for projected volume kube-api-access-w888x for pod openshift-network-diagnostics/network-check-target-2r4wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:23.427139 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.426757    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x podName:63e67d89-c759-424a-9e4f-0c598af63919 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:27.426736692 +0000 UTC m=+10.010168868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-w888x" (UniqueName: "kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x") pod "network-check-target-2r4wh" (UID: "63e67d89-c759-424a-9e4f-0c598af63919") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:23.527144 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:23.527109    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:23.527321 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:23.527161    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:23.527321 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.527279    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:23.527446 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.527335    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs podName:503853d6-c3ef-4994-9151-a4517a4c0abb nodeName:}" failed. No retries permitted until 2026-07-09 18:59:27.527317084 +0000 UTC m=+10.110749253 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs") pod "network-metrics-daemon-srwqt" (UID: "503853d6-c3ef-4994-9151-a4517a4c0abb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:23.527755 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.527734    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:23.527822 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.527791    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret podName:ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:27.527776733 +0000 UTC m=+10.111208896 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret") pod "global-pull-secret-syncer-9tnc2" (UID: "ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:23.873746 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:23.873649    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:23.873912 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.873806    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:23.874208 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:23.874176    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:23.874325 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.874289    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:23.874388 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:23.874355    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:23.874442 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:23.874423    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:25.874435 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:25.872800    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:25.874435 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:25.872994    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:25.874435 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:25.873361    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:25.874435 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:25.873474    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:25.874435 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:25.873658    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:25.874435 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:25.873755    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:27.461117 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:27.461082    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:27.461575 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.461307    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:59:27.461575 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.461330    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:59:27.461575 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.461343    2569 projected.go:196] Error preparing data for projected volume kube-api-access-w888x for pod openshift-network-diagnostics/network-check-target-2r4wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:27.461575 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.461413    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x podName:63e67d89-c759-424a-9e4f-0c598af63919 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:35.46139396 +0000 UTC m=+18.044826120 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-w888x" (UniqueName: "kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x") pod "network-check-target-2r4wh" (UID: "63e67d89-c759-424a-9e4f-0c598af63919") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:27.562097 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:27.562051    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:27.562097 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:27.562104    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:27.562341 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.562243    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:27.562341 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.562291    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs podName:503853d6-c3ef-4994-9151-a4517a4c0abb nodeName:}" failed. No retries permitted until 2026-07-09 18:59:35.562276468 +0000 UTC m=+18.145708639 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs") pod "network-metrics-daemon-srwqt" (UID: "503853d6-c3ef-4994-9151-a4517a4c0abb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:27.562573 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.562558    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:27.562654 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.562596    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret podName:ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:35.562584231 +0000 UTC m=+18.146016389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret") pod "global-pull-secret-syncer-9tnc2" (UID: "ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:27.872888 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:27.872803    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:27.872888 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:27.872824    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:27.873942 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.873609    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:27.873942 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.873659    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:27.873942 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:27.873700    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:27.873942 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:27.873777    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:29.873417 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:29.873367    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:29.873912 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:29.873504    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:29.873912 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:29.873874    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:29.874030 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:29.873953    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:29.874030 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:29.873991    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:29.874138 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:29.874046    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:31.871879 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:31.871794    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:31.872297 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:31.871795    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:31.872297 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:31.871933    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:31.872297 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:31.871795    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:31.872297 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:31.872031    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:31.872297 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:31.872093    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:33.871748 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:33.871670    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:33.871748 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:33.871726    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:33.872233 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:33.871694    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:33.872233 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:33.871822    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:33.872233 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:33.871923    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:33.872233 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:33.872000    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:35.523522 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:35.523484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:35.523916 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.523645    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:59:35.523916 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.523661    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:59:35.523916 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.523671    2569 projected.go:196] Error preparing data for projected volume kube-api-access-w888x for pod openshift-network-diagnostics/network-check-target-2r4wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:35.523916 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.523743    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x podName:63e67d89-c759-424a-9e4f-0c598af63919 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:51.523727546 +0000 UTC m=+34.107159706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-w888x" (UniqueName: "kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x") pod "network-check-target-2r4wh" (UID: "63e67d89-c759-424a-9e4f-0c598af63919") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:35.624644 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:35.624615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:35.624771 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:35.624659    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:35.624825 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.624792    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:35.624825 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.624799    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:35.624914 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.624854    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret podName:ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:51.624840202 +0000 UTC m=+34.208272361 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret") pod "global-pull-secret-syncer-9tnc2" (UID: "ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:35.624914 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.624870    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs podName:503853d6-c3ef-4994-9151-a4517a4c0abb nodeName:}" failed. No retries permitted until 2026-07-09 18:59:51.624862948 +0000 UTC m=+34.208295106 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs") pod "network-metrics-daemon-srwqt" (UID: "503853d6-c3ef-4994-9151-a4517a4c0abb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:35.871587 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:35.871513    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:35.871587 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:35.871542    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:35.871779 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:35.871513    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:35.871779 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.871642    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:35.871779 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.871751    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:35.871875 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:35.871849    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:37.872903 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:37.872871    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:37.873390 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:37.872983    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:37.873390 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:37.873346    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:37.873490 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:37.873456    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:37.873537 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:37.873494    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:37.873587 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:37.873556    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:39.874915 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:39.873702    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:39.874915 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:39.873828    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:39.874915 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:39.874079    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:39.874915 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:39.874181    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:39.874915 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:39.874252    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:39.874915 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:39.874315    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:39.964523 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:39.964315    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-f5bwn" event={"ID":"5bb527e7-82f3-4a04-ac64-789566abc7ac","Type":"ContainerStarted","Data":"049a79e5e750faa7a5d41b8f7bd9dca1421235909bf1ab3919ab39f731e8a64b"}
Jul 09 18:59:39.965709 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:39.965666    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x48nr" event={"ID":"08bf4c59-eb78-4b76-ac07-eefab77990bd","Type":"ContainerStarted","Data":"84939ae3f1c99aab0bdf117fd9d822d0b408d53bf64bb04b0d10ad0e8912bb8b"}
Jul 09 18:59:39.966952 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:39.966933    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gftdt" event={"ID":"9524d05b-67d4-4944-9403-1d5d9a9e7872","Type":"ContainerStarted","Data":"87596022a4ed8cb72fa0afaf492f541e34155fffbbd796a7356e74d8bb79e627"}
Jul 09 18:59:39.985866 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:39.985833    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-f5bwn" podStartSLOduration=2.964417128 podStartE2EDuration="21.985822369s" totalImagesPullingTime="19.021405241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:39.985474552 +0000 UTC m=+22.568906733" watchObservedRunningTime="2026-07-09 18:59:39.985822369 +0000 UTC m=+22.569254548"
Jul 09 18:59:40.016172 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.016128    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gftdt" podStartSLOduration=4.000716335 podStartE2EDuration="23.016105408s" totalImagesPullingTime="19.015389073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:40.000741378 +0000 UTC m=+22.584173559" watchObservedRunningTime="2026-07-09 18:59:40.016105408 +0000 UTC m=+22.599537577"
Jul 09 18:59:40.016821 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.016785    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x48nr" podStartSLOduration=3.02705865 podStartE2EDuration="22.016772731s" totalImagesPullingTime="18.989714081s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:40.016217993 +0000 UTC m=+22.599650194" watchObservedRunningTime="2026-07-09 18:59:40.016772731 +0000 UTC m=+22.600204911"
Jul 09 18:59:40.039673 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.039654    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:40.040200 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.040177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:40.972165 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.971998    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"5369c858d560dff7aa17281efc7362c6ce1b0be1e17d450c41cd2e197e32ddc6"}
Jul 09 18:59:40.973952 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.973759    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77" event={"ID":"a46e7512-0450-4b03-8ff0-a9e93d82166f","Type":"ContainerStarted","Data":"2f260dabb4134ae3132b6b1784de79fcd09c9ef8af490fee696e7ec76d152e82"}
Jul 09 18:59:40.975473 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.975446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-grm29" event={"ID":"060e1b6b-c1bd-4c2f-a967-2635e32c69de","Type":"ContainerStarted","Data":"7369388afe2c9df4a1584654b47588f6be633e8b87dd1f2abb13673ac99570cc"}
Jul 09 18:59:40.977529 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.977503    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-6z8dd" event={"ID":"20730733-2d51-4667-acf3-374120aa43c5","Type":"ContainerStarted","Data":"0be97be4e0dfacb861980177ecfa5b583eaaef41931bccefc213ee2285db9936"}
Jul 09 18:59:40.979352 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.979330    2569 generic.go:356] "Generic (PLEG): container finished" podID="feeda688-dbc8-4a69-8c1c-e7487aad15ba" containerID="85de3f334ba32e12f6f8386c1eaf947645a38f007960fa8d185a8c525aa8f767" exitCode=0
Jul 09 18:59:40.979451 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.979430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerDied","Data":"85de3f334ba32e12f6f8386c1eaf947645a38f007960fa8d185a8c525aa8f767"}
Jul 09 18:59:40.980519 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.980468    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:40.981099 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.981057    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x48nr"
Jul 09 18:59:40.995832 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:40.995794    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-grm29" podStartSLOduration=4.979289943 podStartE2EDuration="23.995780429s" totalImagesPullingTime="19.016490486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:40.995445814 +0000 UTC m=+23.578877995" watchObservedRunningTime="2026-07-09 18:59:40.995780429 +0000 UTC m=+23.579212609"
Jul 09 18:59:41.057256 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.057181    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-6z8dd" podStartSLOduration=4.9975183 podStartE2EDuration="24.057167622s" totalImagesPullingTime="19.059649322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:41.05673864 +0000 UTC m=+23.640170820" watchObservedRunningTime="2026-07-09 18:59:41.057167622 +0000 UTC m=+23.640599802"
Jul 09 18:59:41.410658 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.410623    2569 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 09 18:59:41.831349 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.831015    2569 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-09T18:59:41.410650919Z","UUID":"f3df1937-7a2d-4414-81d9-3d74bf82d5a2","Handler":null,"Name":"","Endpoint":""}
Jul 09 18:59:41.832740 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.832717    2569 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 09 18:59:41.832865 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.832758    2569 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 09 18:59:41.871129 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.871099    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:41.871302 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.871101    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:41.871302 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:41.871230    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:41.871417 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:41.871337    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:41.871417 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.871101    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:41.871522 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:41.871448    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:41.983213 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.983173    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nw8ld" event={"ID":"211d2466-c017-42ae-a8f7-8321c52f6c66","Type":"ContainerStarted","Data":"d933c11349107ee17be9ae3c5ddfbb67ad209260093e21166fbb2f797cef678e"}
Jul 09 18:59:41.987452 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.987163    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"e96dc9b536561308baeefc3f405878fe2010a7e0ed177038a8751edca1c21e8a"}
Jul 09 18:59:41.987452 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.987233    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"1256d63bdefe17a0ba174f55b93942e8ebf5f194fbed5e6e8a0c14ec886a70db"}
Jul 09 18:59:41.987452 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.987274    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"f21b71f9b5ccbdac9220383b5cbb66c22472e61d2b0968e8d037c0f47dd2f970"}
Jul 09 18:59:41.987452 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.987334    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"e48788aac1417a72934a6cfc08fc85b05d51129021d6c9eb3f63ce76c63801c6"}
Jul 09 18:59:41.987452 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.987390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"0cdf3885d3ec84eb87c9ed2db892d17091d07041bcf11a4c39a514427e1e0381"}
Jul 09 18:59:41.989120 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.989091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77" event={"ID":"a46e7512-0450-4b03-8ff0-a9e93d82166f","Type":"ContainerStarted","Data":"b17837a163c791ee6f157dd3901ad1bf5eb38ba7a3f5455039235bd80ba945cb"}
Jul 09 18:59:41.999614 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:41.999565    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nw8ld" podStartSLOduration=5.008288074 podStartE2EDuration="23.999548109s" totalImagesPullingTime="18.991260035s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:41.999404043 +0000 UTC m=+24.582836222" watchObservedRunningTime="2026-07-09 18:59:41.999548109 +0000 UTC m=+24.582980290"
Jul 09 18:59:43.874903 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:43.874874    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:43.875425 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:43.874987    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:43.875425 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:43.875061    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:43.875425 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:43.875104    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:43.875425 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:43.875212    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:43.875425 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:43.875309    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:43.997218 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:43.997130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77" event={"ID":"a46e7512-0450-4b03-8ff0-a9e93d82166f","Type":"ContainerStarted","Data":"bcc6564903037927f1c757bdf0503f25214e76a1af674f917dca958bd068344c"}
Jul 09 18:59:44.017415 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:44.017350    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-p4j77" podStartSLOduration=5.276867793 podStartE2EDuration="27.017330568s" totalImagesPullingTime="21.740462775s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:44.01664684 +0000 UTC m=+26.600079039" watchObservedRunningTime="2026-07-09 18:59:44.017330568 +0000 UTC m=+26.600762749"
Jul 09 18:59:45.871324 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:45.871287    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:45.871960 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:45.871300    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:45.871960 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:45.871406    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:45.871960 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:45.871408    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:45.871960 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:45.871491    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:45.871960 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:45.871564    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:46.003554 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:46.003473    2569 generic.go:356] "Generic (PLEG): container finished" podID="feeda688-dbc8-4a69-8c1c-e7487aad15ba" containerID="9bcefd0bb6fa9923699f31c0687ef647a323c7ebc87e75cbe98e3d317fd6c23b" exitCode=0
Jul 09 18:59:46.003804 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:46.003555    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerDied","Data":"9bcefd0bb6fa9923699f31c0687ef647a323c7ebc87e75cbe98e3d317fd6c23b"}
Jul 09 18:59:46.007070 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:46.007050    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"21ae168f969d9ef7902274fd7ef81710b23745854769d0a3bceeb797e15ea092"}
Jul 09 18:59:47.872857 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:47.872603    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:47.873288 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:47.872662    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:47.873288 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:47.872944    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:47.873288 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:47.872698    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:47.873288 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:47.873041    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:47.873288 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:47.873072    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:48.013930 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:48.013900    2569 generic.go:356] "Generic (PLEG): container finished" podID="feeda688-dbc8-4a69-8c1c-e7487aad15ba" containerID="4d803b5ec2dc523c4eb8d708d855cabcc3b00f4e7bbe1a88137865a30a92189d" exitCode=0
Jul 09 18:59:48.014136 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:48.013986    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerDied","Data":"4d803b5ec2dc523c4eb8d708d855cabcc3b00f4e7bbe1a88137865a30a92189d"}
Jul 09 18:59:48.017735 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:48.017716    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" event={"ID":"2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73","Type":"ContainerStarted","Data":"28b0bd5dd00d8bf3a121c50461ed7dd9f702a87ea8c19fbe863963bc5b9f5fe7"}
Jul 09 18:59:48.017975 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:48.017954    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:48.018137 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:48.018002    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:48.034617 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:48.034596    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:48.067412 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:48.067374    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n" podStartSLOduration=10.199107464 podStartE2EDuration="30.067361004s" totalImagesPullingTime="19.86825354s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:48.067327012 +0000 UTC m=+30.650759215" watchObservedRunningTime="2026-07-09 18:59:48.067361004 +0000 UTC m=+30.650793183"
Jul 09 18:59:49.023872 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.023828    2569 generic.go:356] "Generic (PLEG): container finished" podID="feeda688-dbc8-4a69-8c1c-e7487aad15ba" containerID="d7d57d95646368a00912657b81457b9b4144e6dd8922af8fba1af9a93535efd2" exitCode=0
Jul 09 18:59:49.024488 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.024067    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerDied","Data":"d7d57d95646368a00912657b81457b9b4144e6dd8922af8fba1af9a93535efd2"}
Jul 09 18:59:49.024973 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.024744    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:49.042124 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.042102    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 18:59:49.156868 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.156837    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9tnc2"]
Jul 09 18:59:49.157043 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.156995    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:49.157185 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:49.157095    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:49.159502 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.159478    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2r4wh"]
Jul 09 18:59:49.159602 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.159567    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:49.159655 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:49.159639    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:49.169518 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.169491    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-srwqt"]
Jul 09 18:59:49.169620 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:49.169594    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:49.169722 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:49.169697    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:50.871421 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:50.871388    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:50.872024 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:50.871500    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:50.872024 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:50.871510    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:50.872024 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:50.871532    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:50.872024 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:50.871634    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:50.872024 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:50.871752    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:51.547882 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:51.547844    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:51.548033 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.548008    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 09 18:59:51.548033 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.548032    2569 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 09 18:59:51.548105 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.548043    2569 projected.go:196] Error preparing data for projected volume kube-api-access-w888x for pod openshift-network-diagnostics/network-check-target-2r4wh: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:51.548105 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.548097    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x podName:63e67d89-c759-424a-9e4f-0c598af63919 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:23.548079951 +0000 UTC m=+66.131512109 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-w888x" (UniqueName: "kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x") pod "network-check-target-2r4wh" (UID: "63e67d89-c759-424a-9e4f-0c598af63919") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 09 18:59:51.648456 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:51.648425    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:51.648456 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:51.648462    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:51.648659 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.648567    2569 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:51.648659 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.648592    2569 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:51.648659 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.648624    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs podName:503853d6-c3ef-4994-9151-a4517a4c0abb nodeName:}" failed. No retries permitted until 2026-07-09 19:00:23.648606897 +0000 UTC m=+66.232039063 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs") pod "network-metrics-daemon-srwqt" (UID: "503853d6-c3ef-4994-9151-a4517a4c0abb") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 09 18:59:51.648659 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:51.648656    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret podName:ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc nodeName:}" failed. No retries permitted until 2026-07-09 19:00:23.648637339 +0000 UTC m=+66.232069521 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret") pod "global-pull-secret-syncer-9tnc2" (UID: "ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc") : object "kube-system"/"original-pull-secret" not registered
Jul 09 18:59:52.872044 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:52.872014    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:52.872596 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:52.872055    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:52.872596 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:52.872189    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:52.872596 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:52.872192    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-2r4wh" podUID="63e67d89-c759-424a-9e4f-0c598af63919"
Jul 09 18:59:52.872596 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:52.872280    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-srwqt" podUID="503853d6-c3ef-4994-9151-a4517a4c0abb"
Jul 09 18:59:52.872596 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:52.872358    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9tnc2" podUID="ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc"
Jul 09 18:59:53.731102 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.731033    2569 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-157.ec2.internal" event="NodeReady"
Jul 09 18:59:53.731264 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.731180    2569 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 09 18:59:53.773298 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.773268    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-27zql"]
Jul 09 18:59:53.793477 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.793453    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-hx5wk"]
Jul 09 18:59:53.793612 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.793571    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:53.797064 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.797038    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 09 18:59:53.797203 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.797140    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ssx5x\""
Jul 09 18:59:53.797203 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.797174    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 09 18:59:53.809514 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.809494    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-27zql"]
Jul 09 18:59:53.809610 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.809528    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hx5wk"]
Jul 09 18:59:53.809610 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.809598    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:53.812719 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.812698    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 09 18:59:53.812719 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.812714    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 09 18:59:53.812874 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.812716    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 09 18:59:53.812874 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.812778    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cqw8d\""
Jul 09 18:59:53.964833 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.964809    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-tmp-dir\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:53.965161 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.964913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl2tb\" (UniqueName: \"kubernetes.io/projected/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-kube-api-access-gl2tb\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:53.965161 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.964955    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmvwq\" (UniqueName: \"kubernetes.io/projected/51fc951b-adc4-4219-9e6c-0865ff0e8b61-kube-api-access-fmvwq\") pod \"ingress-canary-hx5wk\" (UID: \"51fc951b-adc4-4219-9e6c-0865ff0e8b61\") " pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:53.965161 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.964975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-config-volume\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:53.965161 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.965025    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:53.965161 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:53.965076    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51fc951b-adc4-4219-9e6c-0865ff0e8b61-cert\") pod \"ingress-canary-hx5wk\" (UID: \"51fc951b-adc4-4219-9e6c-0865ff0e8b61\") " pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:54.040306 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.040200    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerStarted","Data":"a7fbc7767cbfd0ad9cc08cba4d035755c5d73a43ca694b7c9aadbc03ba3ba56f"}
Jul 09 18:59:54.065995 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.065968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-tmp-dir\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.066136 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.066052    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl2tb\" (UniqueName: \"kubernetes.io/projected/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-kube-api-access-gl2tb\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.066136 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.066075    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmvwq\" (UniqueName: \"kubernetes.io/projected/51fc951b-adc4-4219-9e6c-0865ff0e8b61-kube-api-access-fmvwq\") pod \"ingress-canary-hx5wk\" (UID: \"51fc951b-adc4-4219-9e6c-0865ff0e8b61\") " pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:54.066136 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.066096    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-config-volume\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.066311 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.066225    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.066311 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.066277    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51fc951b-adc4-4219-9e6c-0865ff0e8b61-cert\") pod \"ingress-canary-hx5wk\" (UID: \"51fc951b-adc4-4219-9e6c-0865ff0e8b61\") " pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:54.066399 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:54.066378    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:59:54.066468 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.066445    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-tmp-dir\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.066522 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:54.066457    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls podName:64b89e42-7a99-4b6f-afcf-9f3315c8c84c nodeName:}" failed. No retries permitted until 2026-07-09 18:59:54.566423394 +0000 UTC m=+37.149855569 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls") pod "dns-default-27zql" (UID: "64b89e42-7a99-4b6f-afcf-9f3315c8c84c") : secret "dns-default-metrics-tls" not found
Jul 09 18:59:54.066718 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.066677    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-config-volume\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.070533 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.070407    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/51fc951b-adc4-4219-9e6c-0865ff0e8b61-cert\") pod \"ingress-canary-hx5wk\" (UID: \"51fc951b-adc4-4219-9e6c-0865ff0e8b61\") " pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:54.074361 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.074340    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmvwq\" (UniqueName: \"kubernetes.io/projected/51fc951b-adc4-4219-9e6c-0865ff0e8b61-kube-api-access-fmvwq\") pod \"ingress-canary-hx5wk\" (UID: \"51fc951b-adc4-4219-9e6c-0865ff0e8b61\") " pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:54.074443 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.074368    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl2tb\" (UniqueName: \"kubernetes.io/projected/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-kube-api-access-gl2tb\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.118165 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.118142    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-hx5wk"
Jul 09 18:59:54.282274 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.282187    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-hx5wk"]
Jul 09 18:59:54.285799 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:54.285772    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51fc951b_adc4_4219_9e6c_0865ff0e8b61.slice/crio-d4eac5c0d8b5fec939d93905d2991ac4689539c2b8a5d223f7ce52b636c976ee WatchSource:0}: Error finding container d4eac5c0d8b5fec939d93905d2991ac4689539c2b8a5d223f7ce52b636c976ee: Status 404 returned error can't find the container with id d4eac5c0d8b5fec939d93905d2991ac4689539c2b8a5d223f7ce52b636c976ee
Jul 09 18:59:54.569987 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.569906    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:54.570111 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:54.570020    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:59:54.570111 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:54.570072    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls podName:64b89e42-7a99-4b6f-afcf-9f3315c8c84c nodeName:}" failed. No retries permitted until 2026-07-09 18:59:55.570059425 +0000 UTC m=+38.153491583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls") pod "dns-default-27zql" (UID: "64b89e42-7a99-4b6f-afcf-9f3315c8c84c") : secret "dns-default-metrics-tls" not found
Jul 09 18:59:54.872074 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.872039    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 18:59:54.872229 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.872187    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 18:59:54.872489 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.872464    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 18:59:54.875601 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.875580    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 09 18:59:54.877188 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.876891    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 09 18:59:54.877188 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.876891    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dlpdq\""
Jul 09 18:59:54.877188 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.876891    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 18:59:54.877188 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.876983    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8ctlj\""
Jul 09 18:59:54.877188 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:54.876941    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 18:59:55.045564 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:55.045529    2569 generic.go:356] "Generic (PLEG): container finished" podID="feeda688-dbc8-4a69-8c1c-e7487aad15ba" containerID="a7fbc7767cbfd0ad9cc08cba4d035755c5d73a43ca694b7c9aadbc03ba3ba56f" exitCode=0
Jul 09 18:59:55.046036 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:55.045619    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerDied","Data":"a7fbc7767cbfd0ad9cc08cba4d035755c5d73a43ca694b7c9aadbc03ba3ba56f"}
Jul 09 18:59:55.047315 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:55.047294    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hx5wk" event={"ID":"51fc951b-adc4-4219-9e6c-0865ff0e8b61","Type":"ContainerStarted","Data":"d4eac5c0d8b5fec939d93905d2991ac4689539c2b8a5d223f7ce52b636c976ee"}
Jul 09 18:59:55.578947 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:55.578864    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:55.579110 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:55.579039    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:59:55.579157 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:55.579111    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls podName:64b89e42-7a99-4b6f-afcf-9f3315c8c84c nodeName:}" failed. No retries permitted until 2026-07-09 18:59:57.57909386 +0000 UTC m=+40.162526019 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls") pod "dns-default-27zql" (UID: "64b89e42-7a99-4b6f-afcf-9f3315c8c84c") : secret "dns-default-metrics-tls" not found
Jul 09 18:59:56.053626 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.053596    2569 generic.go:356] "Generic (PLEG): container finished" podID="feeda688-dbc8-4a69-8c1c-e7487aad15ba" containerID="a9ff4293fc5c87a190b77d95ac24d6048ca5791e1564bc3c09cf6e0c25f1324c" exitCode=0
Jul 09 18:59:56.054020 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.053659    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerDied","Data":"a9ff4293fc5c87a190b77d95ac24d6048ca5791e1564bc3c09cf6e0c25f1324c"}
Jul 09 18:59:56.276917 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.276882    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"]
Jul 09 18:59:56.280801 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.280231    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"
Jul 09 18:59:56.284643 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.284624    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:59:56.285386 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.285355    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 09 18:59:56.285578 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.285564    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-q7wx4\""
Jul 09 18:59:56.303922 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.303716    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"]
Jul 09 18:59:56.305977 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.305631    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.309476 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.309429    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 09 18:59:56.309476 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.309459    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vhf69\""
Jul 09 18:59:56.309764 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.309742    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 09 18:59:56.310103 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.310083    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 09 18:59:56.310193 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.310086    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:59:56.317421 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.317398    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"]
Jul 09 18:59:56.334666 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.334641    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"]
Jul 09 18:59:56.384363 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.384342    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qlqf\" (UniqueName: \"kubernetes.io/projected/7380ebb1-08f3-4f39-a0fd-ca95b0e1a047-kube-api-access-2qlqf\") pod \"volume-data-source-validator-78495597f6-g9698\" (UID: \"7380ebb1-08f3-4f39-a0fd-ca95b0e1a047\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"
Jul 09 18:59:56.470049 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.470018    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"]
Jul 09 18:59:56.471915 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.471889    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"
Jul 09 18:59:56.472942 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.472916    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"]
Jul 09 18:59:56.474744 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.474727    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"]
Jul 09 18:59:56.474900 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.474881    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:56.474900 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.474895    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-znc8k\""
Jul 09 18:59:56.476279 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.476259    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6b7779465d-kmdjx"]
Jul 09 18:59:56.476467 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.476449    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.478092 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.478073    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 09 18:59:56.478092 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.478085    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g8mwl\""
Jul 09 18:59:56.478231 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.478111    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 09 18:59:56.478231 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.478125    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-695c9bfbc7-mxtgd"]
Jul 09 18:59:56.478313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.478244    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.478533 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.478513    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 09 18:59:56.480411 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.480381    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.482393 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482119    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 09 18:59:56.482393 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482149    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 09 18:59:56.482393 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482231    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 09 18:59:56.482393 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482240    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 09 18:59:56.482393 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482250    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 09 18:59:56.482393 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482298    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 09 18:59:56.482393 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482364    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 09 18:59:56.482746 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482405    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qx286\""
Jul 09 18:59:56.482746 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482483    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6v84g\""
Jul 09 18:59:56.482746 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482535    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 09 18:59:56.482746 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.482553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 09 18:59:56.483612 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.483595    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 09 18:59:56.484590 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.484570    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 09 18:59:56.484676 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.484643    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 09 18:59:56.484875 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.484819    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfzgd\" (UniqueName: \"kubernetes.io/projected/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-kube-api-access-dfzgd\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.484875 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.484870    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-chfdb\""
Jul 09 18:59:56.485009 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.484894    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.485068 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.485010    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.485068 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.485050    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qlqf\" (UniqueName: \"kubernetes.io/projected/7380ebb1-08f3-4f39-a0fd-ca95b0e1a047-kube-api-access-2qlqf\") pod \"volume-data-source-validator-78495597f6-g9698\" (UID: \"7380ebb1-08f3-4f39-a0fd-ca95b0e1a047\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"
Jul 09 18:59:56.486438 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.486408    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 09 18:59:56.492200 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.492180    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"]
Jul 09 18:59:56.493993 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.493431    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"]
Jul 09 18:59:56.501823 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.500846    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"]
Jul 09 18:59:56.504149 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.504122    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b7779465d-kmdjx"]
Jul 09 18:59:56.504272 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.504180    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-695c9bfbc7-mxtgd"]
Jul 09 18:59:56.507773 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.507492    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 09 18:59:56.508669 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.508597    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qlqf\" (UniqueName: \"kubernetes.io/projected/7380ebb1-08f3-4f39-a0fd-ca95b0e1a047-kube-api-access-2qlqf\") pod \"volume-data-source-validator-78495597f6-g9698\" (UID: \"7380ebb1-08f3-4f39-a0fd-ca95b0e1a047\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"
Jul 09 18:59:56.585878 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.585791    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8t27\" (UniqueName: \"kubernetes.io/projected/c8467136-f179-4e60-8a06-c6c12d69e475-kube-api-access-w8t27\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.585878 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.585836    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-installation-pull-secrets\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.585878 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.585857    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5668efae-c36d-43d1-8c85-4af4d19c1d3e-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.585878 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.585874    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6glqf\" (UniqueName: \"kubernetes.io/projected/5668efae-c36d-43d1-8c85-4af4d19c1d3e-kube-api-access-6glqf\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.586192 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.585903    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.586192 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.585960    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-default-certificate\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.586192 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.585996    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-stats-auth\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.586192 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586014    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:56.586192 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpmnh\" (UniqueName: \"kubernetes.io/projected/000a708a-80c8-4573-8511-a4b4473948fc-kube-api-access-cpmnh\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:56.586192 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586125    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9gxl\" (UniqueName: \"kubernetes.io/projected/aac514a1-4721-40e4-a6cc-698d1fd8b52b-kube-api-access-q9gxl\") pod \"network-check-source-869dcfcdcd-rvwcb\" (UID: \"aac514a1-4721-40e4-a6cc-698d1fd8b52b\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"
Jul 09 18:59:56.586192 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586155    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586200    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ba781041-fcee-4068-9a79-6cef8ed6df0f-ca-trust-extracted\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586302    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74nk4\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-kube-api-access-74nk4\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586331    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-bound-sa-token\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfzgd\" (UniqueName: \"kubernetes.io/projected/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-kube-api-access-dfzgd\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586449    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586473    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.586512 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586497    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-image-registry-private-configuration\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.586959 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586593    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-certificates\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.586959 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.586626    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-trusted-ca\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.587089 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.587070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.588602 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.588573    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.590748 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.590730    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"
Jul 09 18:59:56.594633 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.594608    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfzgd\" (UniqueName: \"kubernetes.io/projected/d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b-kube-api-access-dfzgd\") pod \"kube-storage-version-migrator-operator-799cb7f958-q6nxh\" (UID: \"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.618575 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.618548    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"
Jul 09 18:59:56.687782 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687707    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-bound-sa-token\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.687782 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.687936 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-image-registry-private-configuration\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.687936 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687840    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-certificates\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.687936 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687865    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-trusted-ca\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.687936 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687926    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8t27\" (UniqueName: \"kubernetes.io/projected/c8467136-f179-4e60-8a06-c6c12d69e475-kube-api-access-w8t27\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.688054 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687964    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-installation-pull-secrets\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.688054 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.687994    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5668efae-c36d-43d1-8c85-4af4d19c1d3e-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.688054 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688022    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6glqf\" (UniqueName: \"kubernetes.io/projected/5668efae-c36d-43d1-8c85-4af4d19c1d3e-kube-api-access-6glqf\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.688139 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688064    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-default-certificate\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.688139 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688097    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-stats-auth\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.688139 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688126    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:56.688223 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688159    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpmnh\" (UniqueName: \"kubernetes.io/projected/000a708a-80c8-4573-8511-a4b4473948fc-kube-api-access-cpmnh\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:56.688223 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688185    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9gxl\" (UniqueName: \"kubernetes.io/projected/aac514a1-4721-40e4-a6cc-698d1fd8b52b-kube-api-access-q9gxl\") pod \"network-check-source-869dcfcdcd-rvwcb\" (UID: \"aac514a1-4721-40e4-a6cc-698d1fd8b52b\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"
Jul 09 18:59:56.688223 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.688313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688243    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.688313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688298    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ba781041-fcee-4068-9a79-6cef8ed6df0f-ca-trust-extracted\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.688371 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688341    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74nk4\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-kube-api-access-74nk4\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.688402 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.688372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.688525    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.688604    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls podName:5668efae-c36d-43d1-8c85-4af4d19c1d3e nodeName:}" failed. No retries permitted until 2026-07-09 18:59:57.188583124 +0000 UTC m=+39.772015304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5b4kv" (UID: "5668efae-c36d-43d1-8c85-4af4d19c1d3e") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.688869    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.688938    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:57.18891817 +0000 UTC m=+39.772350329 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : secret "router-metrics-certs-default" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.689307    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:57.189286951 +0000 UTC m=+39.772719129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : configmap references non-existent config key: service-ca.crt
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.689400    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.689442    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls podName:000a708a-80c8-4573-8511-a4b4473948fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:57.18942889 +0000 UTC m=+39.772861067 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls") pod "cluster-samples-operator-65db858894-dhgll" (UID: "000a708a-80c8-4573-8511-a4b4473948fc") : secret "samples-operator-tls" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.689823    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-certificates\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.689867    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5668efae-c36d-43d1-8c85-4af4d19c1d3e-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.690197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-trusted-ca\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.690204    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.690233    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-695c9bfbc7-mxtgd: secret "image-registry-tls" not found
Jul 09 18:59:56.690306 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:56.690287    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls podName:ba781041-fcee-4068-9a79-6cef8ed6df0f nodeName:}" failed. No retries permitted until 2026-07-09 18:59:57.190272145 +0000 UTC m=+39.773704319 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls") pod "image-registry-695c9bfbc7-mxtgd" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f") : secret "image-registry-tls" not found
Jul 09 18:59:56.690975 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.690336    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ba781041-fcee-4068-9a79-6cef8ed6df0f-ca-trust-extracted\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.692091 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.692038    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-stats-auth\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.693331 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.693303    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-installation-pull-secrets\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.694939 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.694917    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-default-certificate\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.695071 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.695055    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-image-registry-private-configuration\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.697998 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.697962    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-bound-sa-token\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.700406 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.700379    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8t27\" (UniqueName: \"kubernetes.io/projected/c8467136-f179-4e60-8a06-c6c12d69e475-kube-api-access-w8t27\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:56.701806 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.701577    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74nk4\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-kube-api-access-74nk4\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:56.701906 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.701761    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6glqf\" (UniqueName: \"kubernetes.io/projected/5668efae-c36d-43d1-8c85-4af4d19c1d3e-kube-api-access-6glqf\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:56.701967 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.701949    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpmnh\" (UniqueName: \"kubernetes.io/projected/000a708a-80c8-4573-8511-a4b4473948fc-kube-api-access-cpmnh\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:56.702447 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.702414    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9gxl\" (UniqueName: \"kubernetes.io/projected/aac514a1-4721-40e4-a6cc-698d1fd8b52b-kube-api-access-q9gxl\") pod \"network-check-source-869dcfcdcd-rvwcb\" (UID: \"aac514a1-4721-40e4-a6cc-698d1fd8b52b\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"
Jul 09 18:59:56.729567 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.729540    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698"]
Jul 09 18:59:56.733659 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:56.733635    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7380ebb1_08f3_4f39_a0fd_ca95b0e1a047.slice/crio-d603e8494167d6e3e5ce285e7a45dae8475ba554085c48f0bb305e335587cae5 WatchSource:0}: Error finding container d603e8494167d6e3e5ce285e7a45dae8475ba554085c48f0bb305e335587cae5: Status 404 returned error can't find the container with id d603e8494167d6e3e5ce285e7a45dae8475ba554085c48f0bb305e335587cae5
Jul 09 18:59:56.757369 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.757339    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh"]
Jul 09 18:59:56.759350 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:56.759327    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d13420_7ca3_4bb9_92bc_8c9c3ec5369b.slice/crio-feb67a0be77da56e95b4d59c58b1a0cdc6ce8e81b4b96efa8416c74b2985bbb7 WatchSource:0}: Error finding container feb67a0be77da56e95b4d59c58b1a0cdc6ce8e81b4b96efa8416c74b2985bbb7: Status 404 returned error can't find the container with id feb67a0be77da56e95b4d59c58b1a0cdc6ce8e81b4b96efa8416c74b2985bbb7
Jul 09 18:59:56.784461 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.784440    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"
Jul 09 18:59:56.905983 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:56.905959    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb"]
Jul 09 18:59:56.908584 ip-10-0-128-157 kubenswrapper[2569]: W0709 18:59:56.908557    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaac514a1_4721_40e4_a6cc_698d1fd8b52b.slice/crio-302425f790f8f62e35633da8d1b2f4d13e9b840d68f57440269d67382b386f7b WatchSource:0}: Error finding container 302425f790f8f62e35633da8d1b2f4d13e9b840d68f57440269d67382b386f7b: Status 404 returned error can't find the container with id 302425f790f8f62e35633da8d1b2f4d13e9b840d68f57440269d67382b386f7b
Jul 09 18:59:57.057166 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.057130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh" event={"ID":"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b","Type":"ContainerStarted","Data":"feb67a0be77da56e95b4d59c58b1a0cdc6ce8e81b4b96efa8416c74b2985bbb7"}
Jul 09 18:59:57.058305 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.058286    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698" event={"ID":"7380ebb1-08f3-4f39-a0fd-ca95b0e1a047","Type":"ContainerStarted","Data":"d603e8494167d6e3e5ce285e7a45dae8475ba554085c48f0bb305e335587cae5"}
Jul 09 18:59:57.059622 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.059599    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-hx5wk" event={"ID":"51fc951b-adc4-4219-9e6c-0865ff0e8b61","Type":"ContainerStarted","Data":"d691b042db20ff84423c28597b6346c4394b633cf224ca33ace0b09e7f7a5a02"}
Jul 09 18:59:57.062798 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.062771    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" event={"ID":"feeda688-dbc8-4a69-8c1c-e7487aad15ba","Type":"ContainerStarted","Data":"ca9ca467d1df1049be19ca042b42114b9c91e336c3a78626eab9b5eb5df3f11d"}
Jul 09 18:59:57.063929 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.063909    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb" event={"ID":"aac514a1-4721-40e4-a6cc-698d1fd8b52b","Type":"ContainerStarted","Data":"302425f790f8f62e35633da8d1b2f4d13e9b840d68f57440269d67382b386f7b"}
Jul 09 18:59:57.077906 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.077857    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-hx5wk" podStartSLOduration=2.026489524 podStartE2EDuration="4.077844539s" totalImagesPullingTime="2.051355015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:57.077360181 +0000 UTC m=+39.660792371" watchObservedRunningTime="2026-07-09 18:59:57.077844539 +0000 UTC m=+39.661276718"
Jul 09 18:59:57.099376 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.099334    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6tmrs" podStartSLOduration=10.049160441 podStartE2EDuration="40.099320738s" totalImagesPullingTime="30.050160297s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:17 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:57.09802524 +0000 UTC m=+39.681457421" watchObservedRunningTime="2026-07-09 18:59:57.099320738 +0000 UTC m=+39.682752919"
Jul 09 18:59:57.193045 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.193013    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:57.193194 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.193159    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:57.193194 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193172    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 18:59:57.193313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.193194    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:57.193313 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.193223    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:57.193313 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193251    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.193232355 +0000 UTC m=+40.776664512 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : secret "router-metrics-certs-default" not found
Jul 09 18:59:57.193313 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193288    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:59:57.193313 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193304    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:59:57.193313 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193317    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-695c9bfbc7-mxtgd: secret "image-registry-tls" not found
Jul 09 18:59:57.193571 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.193343    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:57.193571 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193357    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls podName:000a708a-80c8-4573-8511-a4b4473948fc nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.193341272 +0000 UTC m=+40.776773443 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls") pod "cluster-samples-operator-65db858894-dhgll" (UID: "000a708a-80c8-4573-8511-a4b4473948fc") : secret "samples-operator-tls" not found
Jul 09 18:59:57.193571 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193386    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls podName:ba781041-fcee-4068-9a79-6cef8ed6df0f nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.193377739 +0000 UTC m=+40.776809899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls") pod "image-registry-695c9bfbc7-mxtgd" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f") : secret "image-registry-tls" not found
Jul 09 18:59:57.193571 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193407    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:59:57.193571 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193423    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.193409233 +0000 UTC m=+40.776841410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : configmap references non-existent config key: service-ca.crt
Jul 09 18:59:57.193571 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.193465    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls podName:5668efae-c36d-43d1-8c85-4af4d19c1d3e nodeName:}" failed. No retries permitted until 2026-07-09 18:59:58.19344967 +0000 UTC m=+40.776881833 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5b4kv" (UID: "5668efae-c36d-43d1-8c85-4af4d19c1d3e") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:59:57.596432 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:57.596331    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 18:59:57.596588 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.596495    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 18:59:57.596649 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:57.596598    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls podName:64b89e42-7a99-4b6f-afcf-9f3315c8c84c nodeName:}" failed. No retries permitted until 2026-07-09 19:00:01.596551745 +0000 UTC m=+44.179983917 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls") pod "dns-default-27zql" (UID: "64b89e42-7a99-4b6f-afcf-9f3315c8c84c") : secret "dns-default-metrics-tls" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:58.202637    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:58.202740    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:58.202876    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:58.202911    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:58.202939    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203053    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203066    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-695c9bfbc7-mxtgd: secret "image-registry-tls" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203114    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls podName:ba781041-fcee-4068-9a79-6cef8ed6df0f nodeName:}" failed. No retries permitted until 2026-07-09 19:00:00.203096102 +0000 UTC m=+42.786528264 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls") pod "image-registry-695c9bfbc7-mxtgd" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f") : secret "image-registry-tls" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203215    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203255    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls podName:5668efae-c36d-43d1-8c85-4af4d19c1d3e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:00.203242884 +0000 UTC m=+42.786675042 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5b4kv" (UID: "5668efae-c36d-43d1-8c85-4af4d19c1d3e") : secret "cluster-monitoring-operator-tls" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203422    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203473    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:00.203457929 +0000 UTC m=+42.786890094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : secret "router-metrics-certs-default" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203552    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:00.203541162 +0000 UTC m=+42.786973323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : configmap references non-existent config key: service-ca.crt
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203573    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 18:59:58.203804 ip-10-0-128-157 kubenswrapper[2569]: E0709 18:59:58.203599    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls podName:000a708a-80c8-4573-8511-a4b4473948fc nodeName:}" failed. No retries permitted until 2026-07-09 19:00:00.203591394 +0000 UTC m=+42.787023552 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls") pod "cluster-samples-operator-65db858894-dhgll" (UID: "000a708a-80c8-4573-8511-a4b4473948fc") : secret "samples-operator-tls" not found
Jul 09 18:59:59.071929 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:59.071842    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698" event={"ID":"7380ebb1-08f3-4f39-a0fd-ca95b0e1a047","Type":"ContainerStarted","Data":"313ebf17f1f8fcb80b6348855e7d5cc7ce37501e39e9f2c1da7ed2377f246bce"}
Jul 09 18:59:59.089074 ip-10-0-128-157 kubenswrapper[2569]: I0709 18:59:59.089013    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-g9698" podStartSLOduration=1.694946602 podStartE2EDuration="3.08899319s" totalImagesPullingTime="1.394046588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 18:59:59.086329969 +0000 UTC m=+41.669762149" watchObservedRunningTime="2026-07-09 18:59:59.08899319 +0000 UTC m=+41.672425371"
Jul 09 19:00:00.219393 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:00.219350    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:00.219411    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219516    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219533    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-695c9bfbc7-mxtgd: secret "image-registry-tls" not found
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219562    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:04.21953839 +0000 UTC m=+46.802970554 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : configmap references non-existent config key: service-ca.crt
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219593    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls podName:ba781041-fcee-4068-9a79-6cef8ed6df0f nodeName:}" failed. No retries permitted until 2026-07-09 19:00:04.219583634 +0000 UTC m=+46.803015791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls") pod "image-registry-695c9bfbc7-mxtgd" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f") : secret "image-registry-tls" not found
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:00.219645    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:00.219712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219778    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:00.219787    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219797    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219817    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:04.219805609 +0000 UTC m=+46.803237767 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : secret "router-metrics-certs-default" not found
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219852    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls podName:5668efae-c36d-43d1-8c85-4af4d19c1d3e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:04.219837393 +0000 UTC m=+46.803269565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5b4kv" (UID: "5668efae-c36d-43d1-8c85-4af4d19c1d3e") : secret "cluster-monitoring-operator-tls" not found
Jul 09 19:00:00.219918 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219898    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 19:00:00.220455 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:00.219951    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls podName:000a708a-80c8-4573-8511-a4b4473948fc nodeName:}" failed. No retries permitted until 2026-07-09 19:00:04.219936096 +0000 UTC m=+46.803368260 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls") pod "cluster-samples-operator-65db858894-dhgll" (UID: "000a708a-80c8-4573-8511-a4b4473948fc") : secret "samples-operator-tls" not found
Jul 09 19:00:01.078723 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:01.078626    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb" event={"ID":"aac514a1-4721-40e4-a6cc-698d1fd8b52b","Type":"ContainerStarted","Data":"9819e66807f7a4d8f3f96cb4b78a963399b64a09f153fb3a2eeccf2b07043723"}
Jul 09 19:00:01.080232 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:01.080199    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh" event={"ID":"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b","Type":"ContainerStarted","Data":"17cd146388bc1446c952819ce7f498e6beaf83d4324ce1e9e01b2a3bc0a138a8"}
Jul 09 19:00:01.094481 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:01.094438    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-rvwcb" podStartSLOduration=1.2180666900000001 podStartE2EDuration="5.094423901s" totalImagesPullingTime="3.876357211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:01.093923042 +0000 UTC m=+43.677355248" watchObservedRunningTime="2026-07-09 19:00:01.094423901 +0000 UTC m=+43.677856080"
Jul 09 19:00:01.109906 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:01.109856    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh" podStartSLOduration=1.076447117 podStartE2EDuration="5.109843394s" totalImagesPullingTime="4.033396277s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:01.109388498 +0000 UTC m=+43.692820668" watchObservedRunningTime="2026-07-09 19:00:01.109843394 +0000 UTC m=+43.693275571"
Jul 09 19:00:01.630501 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:01.630453    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 19:00:01.630917 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:01.630632    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 19:00:01.630917 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:01.630713    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls podName:64b89e42-7a99-4b6f-afcf-9f3315c8c84c nodeName:}" failed. No retries permitted until 2026-07-09 19:00:09.630697419 +0000 UTC m=+52.214129577 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls") pod "dns-default-27zql" (UID: "64b89e42-7a99-4b6f-afcf-9f3315c8c84c") : secret "dns-default-metrics-tls" not found
Jul 09 19:00:02.143882 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.143849    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6mcr7"]
Jul 09 19:00:02.164035 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.164008    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mcr7"]
Jul 09 19:00:02.164188 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.164120    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.167072 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.167046    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 09 19:00:02.168329 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.168308    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 09 19:00:02.168492 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.168309    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntcpk\""
Jul 09 19:00:02.168595 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.168362    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 09 19:00:02.171872 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.171855    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 09 19:00:02.235843 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.235808    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.236007 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.235884    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.236007 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.235941    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-crio-socket\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.236007 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.235970    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vjzk\" (UniqueName: \"kubernetes.io/projected/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-api-access-2vjzk\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.236155 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.236013    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-data-volume\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.337406 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.337359    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.337406 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.337415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-crio-socket\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.337435    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vjzk\" (UniqueName: \"kubernetes.io/projected/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-api-access-2vjzk\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.337476    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-data-volume\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.337515    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.337524    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.337590    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.337596    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:02.837574459 +0000 UTC m=+45.421006617 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.337595    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-crio-socket\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.337645 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.337630    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:02.837618516 +0000 UTC m=+45.421050675 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : configmap "kube-rbac-proxy" not found
Jul 09 19:00:02.337992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.337903    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-data-volume\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.352207 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.352179    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vjzk\" (UniqueName: \"kubernetes.io/projected/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-api-access-2vjzk\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.842243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.842206    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.842651 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:02.842304    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:02.842651 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.842330    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:00:02.842651 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.842395    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:03.842376213 +0000 UTC m=+46.425808371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : configmap "kube-rbac-proxy" not found
Jul 09 19:00:02.842651 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.842425    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:02.842651 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:02.842479    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:03.842463327 +0000 UTC m=+46.425895502 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:03.729778 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:03.729734    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f5bwn_5bb527e7-82f3-4a04-ac64-789566abc7ac/dns-node-resolver/0.log"
Jul 09 19:00:03.852041 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:03.852005    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:03.852399 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:03.852104    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:03.852399 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:03.852161    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:03.852399 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:03.852192    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:00:03.852399 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:03.852227    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:05.852212133 +0000 UTC m=+48.435644291 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:03.852399 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:03.852245    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:05.852233421 +0000 UTC m=+48.435665579 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : configmap "kube-rbac-proxy" not found
Jul 09 19:00:04.256410 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.256371    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.256454    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.256478    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.256498    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256527    2569 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.256546    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256588    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:12.25657223 +0000 UTC m=+54.840004389 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : secret "router-metrics-certs-default" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256603    2569 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256625    2569 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256635    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:12.25662032 +0000 UTC m=+54.840052478 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : configmap references non-existent config key: service-ca.crt
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256641    2569 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-695c9bfbc7-mxtgd: secret "image-registry-tls" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256631    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256660    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls podName:000a708a-80c8-4573-8511-a4b4473948fc nodeName:}" failed. No retries permitted until 2026-07-09 19:00:12.256646335 +0000 UTC m=+54.840078510 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls") pod "cluster-samples-operator-65db858894-dhgll" (UID: "000a708a-80c8-4573-8511-a4b4473948fc") : secret "samples-operator-tls" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256674    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls podName:5668efae-c36d-43d1-8c85-4af4d19c1d3e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:12.256666768 +0000 UTC m=+54.840098926 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5b4kv" (UID: "5668efae-c36d-43d1-8c85-4af4d19c1d3e") : secret "cluster-monitoring-operator-tls" not found
Jul 09 19:00:04.256707 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.256718    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls podName:ba781041-fcee-4068-9a79-6cef8ed6df0f nodeName:}" failed. No retries permitted until 2026-07-09 19:00:12.256707991 +0000 UTC m=+54.840140149 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls") pod "image-registry-695c9bfbc7-mxtgd" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f") : secret "image-registry-tls" not found
Jul 09 19:00:04.529068 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.528995    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gftdt_9524d05b-67d4-4944-9403-1d5d9a9e7872/node-ca/0.log"
Jul 09 19:00:04.718624 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.718590    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"]
Jul 09 19:00:04.757657 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.757627    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"]
Jul 09 19:00:04.757833 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.757735    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:04.760666 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.760633    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 09 19:00:04.760857 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.760837    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 09 19:00:04.760928 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.760867    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wbzb6\""
Jul 09 19:00:04.865608 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.865582    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:04.866058 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.865848    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:04.967219 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.967177    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:04.967391 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.967253    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:04.967391 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.967343    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 19:00:04.967465 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:04.967415    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert podName:72ca0d0e-77d2-4516-8a52-da3a36a40f1e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:05.46739892 +0000 UTC m=+48.050831081 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rgg8w" (UID: "72ca0d0e-77d2-4516-8a52-da3a36a40f1e") : secret "networking-console-plugin-cert" not found
Jul 09 19:00:04.967908 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:04.967886    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-nginx-conf\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:05.335400 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:05.335318    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hx5wk_51fc951b-adc4-4219-9e6c-0865ff0e8b61/serve-healthcheck-canary/0.log"
Jul 09 19:00:05.472650 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:05.472617    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:05.472862 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:05.472807    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 19:00:05.472926 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:05.472886    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert podName:72ca0d0e-77d2-4516-8a52-da3a36a40f1e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:06.47286457 +0000 UTC m=+49.056296731 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rgg8w" (UID: "72ca0d0e-77d2-4516-8a52-da3a36a40f1e") : secret "networking-console-plugin-cert" not found
Jul 09 19:00:05.875294 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:05.875264    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:05.875662 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:05.875372    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:05.875662 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:05.875399    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:05.875662 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:05.875451    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:09.875437972 +0000 UTC m=+52.458870131 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:05.875662 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:05.875471    2569 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 09 19:00:05.875662 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:05.875515    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:09.875499994 +0000 UTC m=+52.458932155 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : configmap "kube-rbac-proxy" not found
Jul 09 19:00:06.480856 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:06.480812    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:06.481031 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:06.480960    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 19:00:06.481031 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:06.481025    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert podName:72ca0d0e-77d2-4516-8a52-da3a36a40f1e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:08.481009898 +0000 UTC m=+51.064442056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rgg8w" (UID: "72ca0d0e-77d2-4516-8a52-da3a36a40f1e") : secret "networking-console-plugin-cert" not found
Jul 09 19:00:08.496874 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:08.496839    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:08.497284 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:08.496995    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 19:00:08.497284 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:08.497064    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert podName:72ca0d0e-77d2-4516-8a52-da3a36a40f1e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:12.497048437 +0000 UTC m=+55.080480595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rgg8w" (UID: "72ca0d0e-77d2-4516-8a52-da3a36a40f1e") : secret "networking-console-plugin-cert" not found
Jul 09 19:00:09.705096 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:09.705058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 19:00:09.705613 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:09.705234    2569 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 09 19:00:09.705613 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:09.705298    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls podName:64b89e42-7a99-4b6f-afcf-9f3315c8c84c nodeName:}" failed. No retries permitted until 2026-07-09 19:00:25.705282726 +0000 UTC m=+68.288714884 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls") pod "dns-default-27zql" (UID: "64b89e42-7a99-4b6f-afcf-9f3315c8c84c") : secret "dns-default-metrics-tls" not found
Jul 09 19:00:09.907135 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:09.907098    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:09.907303 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:09.907200    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:09.907376 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:09.907361    2569 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:09.907432 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:09.907423    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls podName:a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:17.907408408 +0000 UTC m=+60.490840567 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-6mcr7" (UID: "a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1") : secret "insights-runtime-extractor-tls" not found
Jul 09 19:00:09.907621 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:09.907600    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:12.328542 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.328504    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 19:00:12.328542 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.328549    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:12.329141 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.328569    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:12.329141 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.328776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 19:00:12.329141 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:12.328813    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle podName:c8467136-f179-4e60-8a06-c6c12d69e475 nodeName:}" failed. No retries permitted until 2026-07-09 19:00:28.328779147 +0000 UTC m=+70.912211320 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle") pod "router-default-6b7779465d-kmdjx" (UID: "c8467136-f179-4e60-8a06-c6c12d69e475") : configmap references non-existent config key: service-ca.crt
Jul 09 19:00:12.329141 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:12.328864    2569 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 09 19:00:12.329141 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.328879    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:12.329141 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:12.328916    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls podName:5668efae-c36d-43d1-8c85-4af4d19c1d3e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:28.328900665 +0000 UTC m=+70.912332837 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-5b4kv" (UID: "5668efae-c36d-43d1-8c85-4af4d19c1d3e") : secret "cluster-monitoring-operator-tls" not found
Jul 09 19:00:12.331156 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.331131    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"image-registry-695c9bfbc7-mxtgd\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") " pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:12.331280 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.331197    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c8467136-f179-4e60-8a06-c6c12d69e475-metrics-certs\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:12.331280 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.331234    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/000a708a-80c8-4573-8511-a4b4473948fc-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-dhgll\" (UID: \"000a708a-80c8-4573-8511-a4b4473948fc\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 19:00:12.392609 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.392572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"
Jul 09 19:00:12.431367 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.431344    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:12.530490 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.530459    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:12.530644 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:12.530629    2569 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 09 19:00:12.530737 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:12.530724    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert podName:72ca0d0e-77d2-4516-8a52-da3a36a40f1e nodeName:}" failed. No retries permitted until 2026-07-09 19:00:20.530702485 +0000 UTC m=+63.114134643 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert") pod "networking-console-plugin-6cf856758c-rgg8w" (UID: "72ca0d0e-77d2-4516-8a52-da3a36a40f1e") : secret "networking-console-plugin-cert" not found
Jul 09 19:00:12.554289 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.554261    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll"]
Jul 09 19:00:12.590246 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:12.590223    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-695c9bfbc7-mxtgd"]
Jul 09 19:00:12.592030 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:12.592005    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba781041_fcee_4068_9a79_6cef8ed6df0f.slice/crio-32ea03a49a8aff6a28b8b93d964a292760898ceaa3be03cba25cdf862ad46c5c WatchSource:0}: Error finding container 32ea03a49a8aff6a28b8b93d964a292760898ceaa3be03cba25cdf862ad46c5c: Status 404 returned error can't find the container with id 32ea03a49a8aff6a28b8b93d964a292760898ceaa3be03cba25cdf862ad46c5c
Jul 09 19:00:13.111060 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:13.111024    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll" event={"ID":"000a708a-80c8-4573-8511-a4b4473948fc","Type":"ContainerStarted","Data":"41ab972410a90e5a960df16a397e460654d211badba157f63a563315272eb536"}
Jul 09 19:00:13.112525 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:13.112496    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd" event={"ID":"ba781041-fcee-4068-9a79-6cef8ed6df0f","Type":"ContainerStarted","Data":"8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588"}
Jul 09 19:00:13.112639 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:13.112539    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd" event={"ID":"ba781041-fcee-4068-9a79-6cef8ed6df0f","Type":"ContainerStarted","Data":"32ea03a49a8aff6a28b8b93d964a292760898ceaa3be03cba25cdf862ad46c5c"}
Jul 09 19:00:13.112639 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:13.112623    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:13.135626 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:13.135563    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd" podStartSLOduration=17.135541835 podStartE2EDuration="17.135541835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:13.134723682 +0000 UTC m=+55.718155859" watchObservedRunningTime="2026-07-09 19:00:13.135541835 +0000 UTC m=+55.718974015"
Jul 09 19:00:16.122154 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:16.122118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll" event={"ID":"000a708a-80c8-4573-8511-a4b4473948fc","Type":"ContainerStarted","Data":"fc89fed3d0697666002d7119919a29a6997c5e2b0ecb4a340587acf174d4ef82"}
Jul 09 19:00:16.122580 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:16.122182    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll" event={"ID":"000a708a-80c8-4573-8511-a4b4473948fc","Type":"ContainerStarted","Data":"58fe88d0a4ed242dccf3eadca6309fb6b4decb980f12258e340b1409d9a81413"}
Jul 09 19:00:16.140577 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:16.140534    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-dhgll" podStartSLOduration=17.670818167 podStartE2EDuration="20.140520182s" totalImagesPullingTime="2.469702015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:16.140100481 +0000 UTC m=+58.723532679" watchObservedRunningTime="2026-07-09 19:00:16.140520182 +0000 UTC m=+58.723952363"
Jul 09 19:00:17.973520 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:17.973473    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:17.975881 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:17.975857    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6mcr7\" (UID: \"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1\") " pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:18.076534 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:18.076504    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ntcpk\""
Jul 09 19:00:18.084015 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:18.084000    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6mcr7"
Jul 09 19:00:18.206705 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:18.206649    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6mcr7"]
Jul 09 19:00:18.209462 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:18.209433    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b84a61_b9ef_4ff2_a5a6_3ff4728125e1.slice/crio-8a03c84b39974910782494ee55088e6ad9e87aaa873d49dd7ed5c08af5e9ff79 WatchSource:0}: Error finding container 8a03c84b39974910782494ee55088e6ad9e87aaa873d49dd7ed5c08af5e9ff79: Status 404 returned error can't find the container with id 8a03c84b39974910782494ee55088e6ad9e87aaa873d49dd7ed5c08af5e9ff79
Jul 09 19:00:19.133257 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:19.133225    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mcr7" event={"ID":"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1","Type":"ContainerStarted","Data":"8a03c84b39974910782494ee55088e6ad9e87aaa873d49dd7ed5c08af5e9ff79"}
Jul 09 19:00:20.596887 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:20.596849    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:20.599902 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:20.599874    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/72ca0d0e-77d2-4516-8a52-da3a36a40f1e-networking-console-plugin-cert\") pod \"networking-console-plugin-6cf856758c-rgg8w\" (UID: \"72ca0d0e-77d2-4516-8a52-da3a36a40f1e\") " pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:20.670520 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:20.670486    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-wbzb6\""
Jul 09 19:00:20.677444 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:20.677408    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"
Jul 09 19:00:21.041162 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:21.041134    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-drt6n"
Jul 09 19:00:22.348856 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:22.348832    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6cf856758c-rgg8w"]
Jul 09 19:00:22.351364 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:22.351339    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72ca0d0e_77d2_4516_8a52_da3a36a40f1e.slice/crio-016fe91fb8144beca73d8fdac15280882a199a76b982721e4b79a96b13eb99e0 WatchSource:0}: Error finding container 016fe91fb8144beca73d8fdac15280882a199a76b982721e4b79a96b13eb99e0: Status 404 returned error can't find the container with id 016fe91fb8144beca73d8fdac15280882a199a76b982721e4b79a96b13eb99e0
Jul 09 19:00:23.148479 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.148442    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w" event={"ID":"72ca0d0e-77d2-4516-8a52-da3a36a40f1e","Type":"ContainerStarted","Data":"016fe91fb8144beca73d8fdac15280882a199a76b982721e4b79a96b13eb99e0"}
Jul 09 19:00:23.149945 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.149922    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mcr7" event={"ID":"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1","Type":"ContainerStarted","Data":"1e35abfda263111a9ae828067e313eaf9e9bf15f1529fb5df220e15ecfd81c68"}
Jul 09 19:00:23.624923 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.624891    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 19:00:23.627481 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.627458    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w888x\" (UniqueName: \"kubernetes.io/projected/63e67d89-c759-424a-9e4f-0c598af63919-kube-api-access-w888x\") pod \"network-check-target-2r4wh\" (UID: \"63e67d89-c759-424a-9e4f-0c598af63919\") " pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 19:00:23.693726 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.693635    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dlpdq\""
Jul 09 19:00:23.703797 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.703768    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 19:00:23.703924 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.703814    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"]
Jul 09 19:00:23.705601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.705583    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.708763 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.708731    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 09 19:00:23.709404 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.709260    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 09 19:00:23.709404 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.709291    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 09 19:00:23.709982 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.709956    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 09 19:00:23.718578 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.718553    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"]
Jul 09 19:00:23.724958 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.724376    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-695c9bfbc7-mxtgd"]
Jul 09 19:00:23.725770 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.725663    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 19:00:23.725933 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.725866    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 19:00:23.731207 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.730224    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 09 19:00:23.731207 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.730242    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 09 19:00:23.739283 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.739242    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc-original-pull-secret\") pod \"global-pull-secret-syncer-9tnc2\" (UID: \"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc\") " pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 19:00:23.739905 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.739883    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/503853d6-c3ef-4994-9151-a4517a4c0abb-metrics-certs\") pod \"network-metrics-daemon-srwqt\" (UID: \"503853d6-c3ef-4994-9151-a4517a4c0abb\") " pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 19:00:23.757056 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.756910    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75f67cc456-6r46r"]
Jul 09 19:00:23.759853 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.759821    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.772257 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.772224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f67cc456-6r46r"]
Jul 09 19:00:23.797214 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.797190    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-gfwmb"]
Jul 09 19:00:23.799098 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.799084    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gfwmb"
Jul 09 19:00:23.802993 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.802967    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 09 19:00:23.802993 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.802986    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 09 19:00:23.803189 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.803161    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-f5ff2\""
Jul 09 19:00:23.817895 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.817870    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gfwmb"]
Jul 09 19:00:23.826658 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826630    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c964eefe-7516-4ef1-852c-52cf0440928f-registry-certificates\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.826788 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826663    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c964eefe-7516-4ef1-852c-52cf0440928f-installation-pull-secrets\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.826788 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826749    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-klusterlet-config\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.826788 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826780    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-tmp\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.826897 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826799    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b47kl\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-kube-api-access-b47kl\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.826897 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826837    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c964eefe-7516-4ef1-852c-52cf0440928f-image-registry-private-configuration\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.826897 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826856    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-registry-tls\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.826897 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826878    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c964eefe-7516-4ef1-852c-52cf0440928f-ca-trust-extracted\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.826897 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826896    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c964eefe-7516-4ef1-852c-52cf0440928f-trusted-ca\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.827054 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826912    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdp7l\" (UniqueName: \"kubernetes.io/projected/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-kube-api-access-mdp7l\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.827054 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.826931    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-bound-sa-token\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.855559 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.855533    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-2r4wh"]
Jul 09 19:00:23.858289 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:23.858269    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63e67d89_c759_424a_9e4f_0c598af63919.slice/crio-de6036e745f8508a27a888509fbba7f86b26a9e7cde6b98400cf71ffc73b4e38 WatchSource:0}: Error finding container de6036e745f8508a27a888509fbba7f86b26a9e7cde6b98400cf71ffc73b4e38: Status 404 returned error can't find the container with id de6036e745f8508a27a888509fbba7f86b26a9e7cde6b98400cf71ffc73b4e38
Jul 09 19:00:23.927806 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.927775    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c964eefe-7516-4ef1-852c-52cf0440928f-image-registry-private-configuration\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.927947 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.927814    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-registry-tls\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.927947 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.927847    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c964eefe-7516-4ef1-852c-52cf0440928f-ca-trust-extracted\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.928052 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928023    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c964eefe-7516-4ef1-852c-52cf0440928f-trusted-ca\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.928105 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928058    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdp7l\" (UniqueName: \"kubernetes.io/projected/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-kube-api-access-mdp7l\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.928105 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928082    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-bound-sa-token\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.928212 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928143    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c964eefe-7516-4ef1-852c-52cf0440928f-registry-certificates\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.928212 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928187    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c964eefe-7516-4ef1-852c-52cf0440928f-installation-pull-secrets\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.928310 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928255    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-klusterlet-config\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.928310 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928265    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c964eefe-7516-4ef1-852c-52cf0440928f-ca-trust-extracted\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.928310 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928288    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9zfg\" (UniqueName: \"kubernetes.io/projected/8c720f71-d20e-4dac-b09f-c8100d4b3ffb-kube-api-access-x9zfg\") pod \"downloads-68c9cbb497-gfwmb\" (UID: \"8c720f71-d20e-4dac-b09f-c8100d4b3ffb\") " pod="openshift-console/downloads-68c9cbb497-gfwmb"
Jul 09 19:00:23.928462 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928323    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-tmp\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.928462 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.928352    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b47kl\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-kube-api-access-b47kl\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.929055 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.929031    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-tmp\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.929175 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.929031    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c964eefe-7516-4ef1-852c-52cf0440928f-trusted-ca\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.929428 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.929394    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c964eefe-7516-4ef1-852c-52cf0440928f-registry-certificates\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.930615 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.930586    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c964eefe-7516-4ef1-852c-52cf0440928f-installation-pull-secrets\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.930835 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.930814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-klusterlet-config\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.930921 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.930861    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-registry-tls\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.931005 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.930987    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c964eefe-7516-4ef1-852c-52cf0440928f-image-registry-private-configuration\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.939972 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.939945    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdp7l\" (UniqueName: \"kubernetes.io/projected/12a90f8a-ae26-4e03-8370-a98a7cbcb7c9-kube-api-access-mdp7l\") pod \"klusterlet-addon-workmgr-794466c978-rdvsd\" (UID: \"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:23.940053 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.939999    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-bound-sa-token\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.940053 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.940029    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b47kl\" (UniqueName: \"kubernetes.io/projected/c964eefe-7516-4ef1-852c-52cf0440928f-kube-api-access-b47kl\") pod \"image-registry-75f67cc456-6r46r\" (UID: \"c964eefe-7516-4ef1-852c-52cf0440928f\") " pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:23.990083 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.990002    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9tnc2"
Jul 09 19:00:23.998653 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:23.998627    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8ctlj\""
Jul 09 19:00:24.006801 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.006774    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-srwqt"
Jul 09 19:00:24.029447 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.029403    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9zfg\" (UniqueName: \"kubernetes.io/projected/8c720f71-d20e-4dac-b09f-c8100d4b3ffb-kube-api-access-x9zfg\") pod \"downloads-68c9cbb497-gfwmb\" (UID: \"8c720f71-d20e-4dac-b09f-c8100d4b3ffb\") " pod="openshift-console/downloads-68c9cbb497-gfwmb"
Jul 09 19:00:24.032385 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.032359    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:24.042646 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.042622    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9zfg\" (UniqueName: \"kubernetes.io/projected/8c720f71-d20e-4dac-b09f-c8100d4b3ffb-kube-api-access-x9zfg\") pod \"downloads-68c9cbb497-gfwmb\" (UID: \"8c720f71-d20e-4dac-b09f-c8100d4b3ffb\") " pod="openshift-console/downloads-68c9cbb497-gfwmb"
Jul 09 19:00:24.080747 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.079909    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:24.111931 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.110672    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-gfwmb"
Jul 09 19:00:24.164421 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.164321    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mcr7" event={"ID":"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1","Type":"ContainerStarted","Data":"78ab8dc0ee4ecd4d8cdbd4a340b268e53a29c94cbe04d98add11983e72d9b45b"}
Jul 09 19:00:24.174154 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.170414    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2r4wh" event={"ID":"63e67d89-c759-424a-9e4f-0c598af63919","Type":"ContainerStarted","Data":"fc2c7e949d098928acfabb59672e60fb44d8b088d35f68e8474cfc20d9dd9119"}
Jul 09 19:00:24.174154 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.170473    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-2r4wh" event={"ID":"63e67d89-c759-424a-9e4f-0c598af63919","Type":"ContainerStarted","Data":"de6036e745f8508a27a888509fbba7f86b26a9e7cde6b98400cf71ffc73b4e38"}
Jul 09 19:00:24.174154 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.171253    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 19:00:24.185558 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.182149    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9tnc2"]
Jul 09 19:00:24.185871 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:24.185645    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae2e2595_c6c5_47e7_88f9_1661b5a3c4fc.slice/crio-705b30a911a41afc39dc1f34786fa01fde12ddccefadca185db5f6772e06da2a WatchSource:0}: Error finding container 705b30a911a41afc39dc1f34786fa01fde12ddccefadca185db5f6772e06da2a: Status 404 returned error can't find the container with id 705b30a911a41afc39dc1f34786fa01fde12ddccefadca185db5f6772e06da2a
Jul 09 19:00:24.192572 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.189270    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-2r4wh" podStartSLOduration=67.189250326 podStartE2EDuration="1m7.189250326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:24.188043357 +0000 UTC m=+66.771475609" watchObservedRunningTime="2026-07-09 19:00:24.189250326 +0000 UTC m=+66.772682507"
Jul 09 19:00:24.225156 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.225119    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-srwqt"]
Jul 09 19:00:24.230637 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:24.230605    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod503853d6_c3ef_4994_9151_a4517a4c0abb.slice/crio-50b1644a53ea1e8fb3b7a2c064192f34ab202b8de912c4c5d21108f8cc011510 WatchSource:0}: Error finding container 50b1644a53ea1e8fb3b7a2c064192f34ab202b8de912c4c5d21108f8cc011510: Status 404 returned error can't find the container with id 50b1644a53ea1e8fb3b7a2c064192f34ab202b8de912c4c5d21108f8cc011510
Jul 09 19:00:24.249929 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.249905    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"]
Jul 09 19:00:24.258146 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:24.258116    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12a90f8a_ae26_4e03_8370_a98a7cbcb7c9.slice/crio-735435df36ebc419a9d925f065a39d6df9a0474323b63b2ca0513154b9868e64 WatchSource:0}: Error finding container 735435df36ebc419a9d925f065a39d6df9a0474323b63b2ca0513154b9868e64: Status 404 returned error can't find the container with id 735435df36ebc419a9d925f065a39d6df9a0474323b63b2ca0513154b9868e64
Jul 09 19:00:24.282925 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.282780    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75f67cc456-6r46r"]
Jul 09 19:00:24.285764 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:24.285737    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc964eefe_7516_4ef1_852c_52cf0440928f.slice/crio-4d19a42728d9ae38a1886123fe3b77578fa9a37f90373bb281b2f8a3ecfa834d WatchSource:0}: Error finding container 4d19a42728d9ae38a1886123fe3b77578fa9a37f90373bb281b2f8a3ecfa834d: Status 404 returned error can't find the container with id 4d19a42728d9ae38a1886123fe3b77578fa9a37f90373bb281b2f8a3ecfa834d
Jul 09 19:00:24.306770 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:24.306744    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-gfwmb"]
Jul 09 19:00:24.318129 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:24.318104    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c720f71_d20e_4dac_b09f_c8100d4b3ffb.slice/crio-8c51e66ccb6c256116357590ba9cb30186d6847f29c1aa1599a7f64a233d140c WatchSource:0}: Error finding container 8c51e66ccb6c256116357590ba9cb30186d6847f29c1aa1599a7f64a233d140c: Status 404 returned error can't find the container with id 8c51e66ccb6c256116357590ba9cb30186d6847f29c1aa1599a7f64a233d140c
Jul 09 19:00:25.178169 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.178052    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w" event={"ID":"72ca0d0e-77d2-4516-8a52-da3a36a40f1e","Type":"ContainerStarted","Data":"c794b42e1908726d0100581cc3f42d1602b5de0386b125b452fe0953d162cf6c"}
Jul 09 19:00:25.180812 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.180738    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gfwmb" event={"ID":"8c720f71-d20e-4dac-b09f-c8100d4b3ffb","Type":"ContainerStarted","Data":"8c51e66ccb6c256116357590ba9cb30186d6847f29c1aa1599a7f64a233d140c"}
Jul 09 19:00:25.183114 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.183091    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f67cc456-6r46r" event={"ID":"c964eefe-7516-4ef1-852c-52cf0440928f","Type":"ContainerStarted","Data":"efad21c11625adb0d9d3514a1f30316250620df282ce0f4e7b32876dd2f09f9b"}
Jul 09 19:00:25.183230 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.183130    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75f67cc456-6r46r" event={"ID":"c964eefe-7516-4ef1-852c-52cf0440928f","Type":"ContainerStarted","Data":"4d19a42728d9ae38a1886123fe3b77578fa9a37f90373bb281b2f8a3ecfa834d"}
Jul 09 19:00:25.183298 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.183248    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:25.184928 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.184866    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd" event={"ID":"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9","Type":"ContainerStarted","Data":"735435df36ebc419a9d925f065a39d6df9a0474323b63b2ca0513154b9868e64"}
Jul 09 19:00:25.186329 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.186303    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-srwqt" event={"ID":"503853d6-c3ef-4994-9151-a4517a4c0abb","Type":"ContainerStarted","Data":"50b1644a53ea1e8fb3b7a2c064192f34ab202b8de912c4c5d21108f8cc011510"}
Jul 09 19:00:25.194821 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.194756    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9tnc2" event={"ID":"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc","Type":"ContainerStarted","Data":"705b30a911a41afc39dc1f34786fa01fde12ddccefadca185db5f6772e06da2a"}
Jul 09 19:00:25.196374 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.196217    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6cf856758c-rgg8w" podStartSLOduration=18.853981081 podStartE2EDuration="21.196184178s" totalImagesPullingTime="2.342203097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:25.195202555 +0000 UTC m=+67.778634760" watchObservedRunningTime="2026-07-09 19:00:25.196184178 +0000 UTC m=+67.779616357"
Jul 09 19:00:25.215358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.215303    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75f67cc456-6r46r" podStartSLOduration=2.21528782 podStartE2EDuration="2.21528782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:25.214575609 +0000 UTC m=+67.798007789" watchObservedRunningTime="2026-07-09 19:00:25.21528782 +0000 UTC m=+67.798719999"
Jul 09 19:00:25.759675 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.759588    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 19:00:25.763602 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.763569    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/64b89e42-7a99-4b6f-afcf-9f3315c8c84c-metrics-tls\") pod \"dns-default-27zql\" (UID: \"64b89e42-7a99-4b6f-afcf-9f3315c8c84c\") " pod="openshift-dns/dns-default-27zql"
Jul 09 19:00:25.906013 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.905802    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ssx5x\""
Jul 09 19:00:25.913850 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:25.913569    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-27zql"
Jul 09 19:00:26.119170 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:26.119004    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-27zql"]
Jul 09 19:00:26.124268 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:26.124233    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64b89e42_7a99_4b6f_afcf_9f3315c8c84c.slice/crio-a4ba73331062ab221032a45eab31efc96f6ace8cc77d0cdeeb10aaa32fd2f7e0 WatchSource:0}: Error finding container a4ba73331062ab221032a45eab31efc96f6ace8cc77d0cdeeb10aaa32fd2f7e0: Status 404 returned error can't find the container with id a4ba73331062ab221032a45eab31efc96f6ace8cc77d0cdeeb10aaa32fd2f7e0
Jul 09 19:00:26.194758 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:26.194717    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-27zql" event={"ID":"64b89e42-7a99-4b6f-afcf-9f3315c8c84c","Type":"ContainerStarted","Data":"a4ba73331062ab221032a45eab31efc96f6ace8cc77d0cdeeb10aaa32fd2f7e0"}
Jul 09 19:00:26.199975 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:26.199947    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-srwqt" event={"ID":"503853d6-c3ef-4994-9151-a4517a4c0abb","Type":"ContainerStarted","Data":"f38470c5abad9f42fd772fa89c7e091ff206ae984734957310ac212bda7954d2"}
Jul 09 19:00:26.200110 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:26.200000    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-srwqt" event={"ID":"503853d6-c3ef-4994-9151-a4517a4c0abb","Type":"ContainerStarted","Data":"e65436679fa4fcad67372aeaacfba95ec6f8fbd1c091ff92296d9619653fb2aa"}
Jul 09 19:00:28.384916 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.384864    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 19:00:28.385312 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.384970    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:28.385787 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.385698    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c8467136-f179-4e60-8a06-c6c12d69e475-service-ca-bundle\") pod \"router-default-6b7779465d-kmdjx\" (UID: \"c8467136-f179-4e60-8a06-c6c12d69e475\") " pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:28.387869 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.387843    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5668efae-c36d-43d1-8c85-4af4d19c1d3e-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-5b4kv\" (UID: \"5668efae-c36d-43d1-8c85-4af4d19c1d3e\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 19:00:28.622135 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.622099    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6v84g\""
Jul 09 19:00:28.628519 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.628496    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-qx286\""
Jul 09 19:00:28.629555 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.629530    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"
Jul 09 19:00:28.636975 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:28.636908    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:32.893926 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:32.893864    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-srwqt" podStartSLOduration=73.748378255 podStartE2EDuration="1m14.893842616s" totalImagesPullingTime="1.145464361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:26.218865837 +0000 UTC m=+68.802298021" watchObservedRunningTime="2026-07-09 19:00:32.893842616 +0000 UTC m=+75.477274803"
Jul 09 19:00:32.895127 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:32.895083    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv"]
Jul 09 19:00:33.293314 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:33.293278    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5668efae_c36d_43d1_8c85_4af4d19c1d3e.slice/crio-3614282b02e09a293393babef16658151e49b4ec26e0f9120faa68cd2fe61547 WatchSource:0}: Error finding container 3614282b02e09a293393babef16658151e49b4ec26e0f9120faa68cd2fe61547: Status 404 returned error can't find the container with id 3614282b02e09a293393babef16658151e49b4ec26e0f9120faa68cd2fe61547
Jul 09 19:00:33.452656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:33.452552    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6b7779465d-kmdjx"]
Jul 09 19:00:33.457235 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:33.457207    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8467136_f179_4e60_8a06_c6c12d69e475.slice/crio-01643c118dd9e16b4d2ee5fb0d495627eb7bb1beff1e8c0b48752d06f5b5ea07 WatchSource:0}: Error finding container 01643c118dd9e16b4d2ee5fb0d495627eb7bb1beff1e8c0b48752d06f5b5ea07: Status 404 returned error can't find the container with id 01643c118dd9e16b4d2ee5fb0d495627eb7bb1beff1e8c0b48752d06f5b5ea07
Jul 09 19:00:33.730560 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:33.730516    2569 patch_prober.go:28] interesting pod/image-registry-695c9bfbc7-mxtgd container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 19:00:33.730741 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:33.730574    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd" podUID="ba781041-fcee-4068-9a79-6cef8ed6df0f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 19:00:34.236380 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.236342    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b7779465d-kmdjx" event={"ID":"c8467136-f179-4e60-8a06-c6c12d69e475","Type":"ContainerStarted","Data":"2a10528464b6067de4f0dc5d7cc378ed72b57f7f689b5e07f3f9efb6b997254c"}
Jul 09 19:00:34.236845 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.236405    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6b7779465d-kmdjx" event={"ID":"c8467136-f179-4e60-8a06-c6c12d69e475","Type":"ContainerStarted","Data":"01643c118dd9e16b4d2ee5fb0d495627eb7bb1beff1e8c0b48752d06f5b5ea07"}
Jul 09 19:00:34.238987 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.238956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6mcr7" event={"ID":"a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1","Type":"ContainerStarted","Data":"bc0fa2e77eb7836e0a1b934811949fb862d65267e38b070e48e1634dadb6d0f5"}
Jul 09 19:00:34.240811 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.240780    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd" event={"ID":"12a90f8a-ae26-4e03-8370-a98a7cbcb7c9","Type":"ContainerStarted","Data":"4222cbd4d9afd9fa8504bae3c36aa5126a3087ad7374ed680506eee5db028eb4"}
Jul 09 19:00:34.240969 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.240943    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:34.242827 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.242808    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd"
Jul 09 19:00:34.242896 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.242846    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9tnc2" event={"ID":"ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc","Type":"ContainerStarted","Data":"7c87c30c8d89d13303eccc00553a19e9b5b3328c6bacebfa1e116605acdb6db6"}
Jul 09 19:00:34.244146 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.244103    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv" event={"ID":"5668efae-c36d-43d1-8c85-4af4d19c1d3e","Type":"ContainerStarted","Data":"3614282b02e09a293393babef16658151e49b4ec26e0f9120faa68cd2fe61547"}
Jul 09 19:00:34.246148 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.246126    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-27zql" event={"ID":"64b89e42-7a99-4b6f-afcf-9f3315c8c84c","Type":"ContainerStarted","Data":"ccb2c0a2eefb2260b8d875cc98c32631c680f8de031d5b5967c581ccee3eb393"}
Jul 09 19:00:34.246244 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.246168    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-27zql" event={"ID":"64b89e42-7a99-4b6f-afcf-9f3315c8c84c","Type":"ContainerStarted","Data":"1785d53da795a82978782c4cdce8498aad12523ae8da01f1f7a7587026ccd9f2"}
Jul 09 19:00:34.246291 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.246263    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-27zql"
Jul 09 19:00:34.260748 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.260697    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6b7779465d-kmdjx" podStartSLOduration=38.260665255 podStartE2EDuration="38.260665255s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:34.259095992 +0000 UTC m=+76.842528173" watchObservedRunningTime="2026-07-09 19:00:34.260665255 +0000 UTC m=+76.844097437"
Jul 09 19:00:34.279740 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.279670    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-27zql" podStartSLOduration=34.068799626 podStartE2EDuration="41.279652715s" totalImagesPullingTime="7.210853089s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:34.278750154 +0000 UTC m=+76.862182356" watchObservedRunningTime="2026-07-09 19:00:34.279652715 +0000 UTC m=+76.863084898"
Jul 09 19:00:34.295818 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.295771    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9tnc2" podStartSLOduration=67.190985267 podStartE2EDuration="1m16.295757252s" totalImagesPullingTime="9.104771985s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 18:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:34.295517282 +0000 UTC m=+76.878949461" watchObservedRunningTime="2026-07-09 19:00:34.295757252 +0000 UTC m=+76.879189425"
Jul 09 19:00:34.328877 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.328818    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6mcr7" podStartSLOduration=17.893992509 podStartE2EDuration="32.328799904s" totalImagesPullingTime="14.434807395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:00:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:34.31304053 +0000 UTC m=+76.896472711" watchObservedRunningTime="2026-07-09 19:00:34.328799904 +0000 UTC m=+76.912232085"
Jul 09 19:00:34.329052 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.328983    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-794466c978-rdvsd" podStartSLOduration=2.295906153 podStartE2EDuration="11.328976027s" totalImagesPullingTime="9.033069874s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:34.32842768 +0000 UTC m=+76.911859861" watchObservedRunningTime="2026-07-09 19:00:34.328976027 +0000 UTC m=+76.912408208"
Jul 09 19:00:34.637407 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.637371    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:34.640427 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:34.640401    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:35.250057 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:35.250025    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:35.251282 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:35.251257    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6b7779465d-kmdjx"
Jul 09 19:00:43.729483 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:43.729446    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:44.086036 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.085994    2569 patch_prober.go:28] interesting pod/image-registry-75f67cc456-6r46r container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 09 19:00:44.086505 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.086475    2569 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-75f67cc456-6r46r" podUID="c964eefe-7516-4ef1-852c-52cf0440928f" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 19:00:44.252370 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.252340    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-27zql"
Jul 09 19:00:44.286525 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.286445    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-gfwmb" event={"ID":"8c720f71-d20e-4dac-b09f-c8100d4b3ffb","Type":"ContainerStarted","Data":"d33f04d7c71f5fe856f52ea186ccae243146fdfc58926a61e7df46f9dde83396"}
Jul 09 19:00:44.287176 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.287152    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-gfwmb"
Jul 09 19:00:44.288985 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.288956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv" event={"ID":"5668efae-c36d-43d1-8c85-4af4d19c1d3e","Type":"ContainerStarted","Data":"593288a65a8d188216bfbd44a8ce3b3985cb80f9938ecb8255def1f08bc6d5d4"}
Jul 09 19:00:44.289450 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.289421    2569 patch_prober.go:28] interesting pod/downloads-68c9cbb497-gfwmb container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.133.0.19:8080/\": dial tcp 10.133.0.19:8080: connect: connection refused" start-of-body=
Jul 09 19:00:44.289569 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.289483    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-68c9cbb497-gfwmb" podUID="8c720f71-d20e-4dac-b09f-c8100d4b3ffb" containerName="download-server" probeResult="failure" output="Get \"http://10.133.0.19:8080/\": dial tcp 10.133.0.19:8080: connect: connection refused"
Jul 09 19:00:44.310993 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.310939    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-gfwmb" podStartSLOduration=1.5515491300000002 podStartE2EDuration="21.310919139s" totalImagesPullingTime="19.759370009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:00:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:44.310847223 +0000 UTC m=+86.894279404" watchObservedRunningTime="2026-07-09 19:00:44.310919139 +0000 UTC m=+86.894351320"
Jul 09 19:00:44.331835 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:44.331792    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-5b4kv" podStartSLOduration=37.598166494 podStartE2EDuration="48.33177514s" totalImagesPullingTime="10.733608646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 18:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:44.330732105 +0000 UTC m=+86.914164311" watchObservedRunningTime="2026-07-09 19:00:44.33177514 +0000 UTC m=+86.915207318"
Jul 09 19:00:45.311962 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:45.311933    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-gfwmb"
Jul 09 19:00:46.205653 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:46.205621    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75f67cc456-6r46r"
Jul 09 19:00:48.752797 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:48.752750    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd" podUID="ba781041-fcee-4068-9a79-6cef8ed6df0f" containerName="registry" containerID="cri-o://8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588" gracePeriod=30
Jul 09 19:00:49.038450 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.038421    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:49.134546 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134514    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-bound-sa-token\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.134744 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134559    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-trusted-ca\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.134744 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134590    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.134744 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134628    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-installation-pull-secrets\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.134744 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134651    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-image-registry-private-configuration\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.134744 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134714    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ba781041-fcee-4068-9a79-6cef8ed6df0f-ca-trust-extracted\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.135016 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134770    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-certificates\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.135016 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.134806    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74nk4\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-kube-api-access-74nk4\") pod \"ba781041-fcee-4068-9a79-6cef8ed6df0f\" (UID: \"ba781041-fcee-4068-9a79-6cef8ed6df0f\") "
Jul 09 19:00:49.135650 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.135356    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:00:49.135650 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.135607    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:00:49.137630 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.137601    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:00:49.137756 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.137645    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:00:49.137756 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.137673    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-kube-api-access-74nk4" (OuterVolumeSpecName: "kube-api-access-74nk4") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "kube-api-access-74nk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:00:49.137879 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.137833    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:00:49.139843 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.139816    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:00:49.145427 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.145401    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba781041-fcee-4068-9a79-6cef8ed6df0f-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "ba781041-fcee-4068-9a79-6cef8ed6df0f" (UID: "ba781041-fcee-4068-9a79-6cef8ed6df0f"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:00:49.235299 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235261    2569 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-bound-sa-token\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.235299 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235293    2569 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-trusted-ca\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.235299 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235301    2569 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-tls\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.235535 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235313    2569 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-installation-pull-secrets\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.235535 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235332    2569 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ba781041-fcee-4068-9a79-6cef8ed6df0f-image-registry-private-configuration\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.235535 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235347    2569 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ba781041-fcee-4068-9a79-6cef8ed6df0f-ca-trust-extracted\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.235535 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235360    2569 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ba781041-fcee-4068-9a79-6cef8ed6df0f-registry-certificates\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.235535 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.235374    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74nk4\" (UniqueName: \"kubernetes.io/projected/ba781041-fcee-4068-9a79-6cef8ed6df0f-kube-api-access-74nk4\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:00:49.315492 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.315399    2569 generic.go:356] "Generic (PLEG): container finished" podID="ba781041-fcee-4068-9a79-6cef8ed6df0f" containerID="8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588" exitCode=0
Jul 09 19:00:49.315492 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.315459    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd" event={"ID":"ba781041-fcee-4068-9a79-6cef8ed6df0f","Type":"ContainerDied","Data":"8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588"}
Jul 09 19:00:49.315492 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.315471    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd"
Jul 09 19:00:49.315823 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.315511    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-695c9bfbc7-mxtgd" event={"ID":"ba781041-fcee-4068-9a79-6cef8ed6df0f","Type":"ContainerDied","Data":"32ea03a49a8aff6a28b8b93d964a292760898ceaa3be03cba25cdf862ad46c5c"}
Jul 09 19:00:49.315823 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.315541    2569 scope.go:117] "RemoveContainer" containerID="8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588"
Jul 09 19:00:49.331866 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.331842    2569 scope.go:117] "RemoveContainer" containerID="8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588"
Jul 09 19:00:49.332176 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:00:49.332150    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588\": container with ID starting with 8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588 not found: ID does not exist" containerID="8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588"
Jul 09 19:00:49.332290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.332187    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588"} err="failed to get container status \"8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588\": rpc error: code = NotFound desc = could not find container \"8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588\": container with ID starting with 8b399881d9c195a033e10b5543187d76e13bab35601088c21c10a3b648c99588 not found: ID does not exist"
Jul 09 19:00:49.342668 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.342642    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-695c9bfbc7-mxtgd"]
Jul 09 19:00:49.346131 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.346106    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-695c9bfbc7-mxtgd"]
Jul 09 19:00:49.877418 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:49.877381    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ba781041-fcee-4068-9a79-6cef8ed6df0f" path="/var/lib/kubelet/pods/ba781041-fcee-4068-9a79-6cef8ed6df0f/volumes"
Jul 09 19:00:53.983940 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:53.983910    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-n6qfs"]
Jul 09 19:00:53.984309 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:53.984198    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba781041-fcee-4068-9a79-6cef8ed6df0f" containerName="registry"
Jul 09 19:00:53.984309 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:53.984210    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba781041-fcee-4068-9a79-6cef8ed6df0f" containerName="registry"
Jul 09 19:00:53.984309 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:53.984255    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba781041-fcee-4068-9a79-6cef8ed6df0f" containerName="registry"
Jul 09 19:00:54.031834 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.031805    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.035473 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.035445    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 09 19:00:54.035638 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.035470    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 09 19:00:54.035826 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.035811    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 09 19:00:54.035906 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.035812    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 09 19:00:54.035906 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.035846    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5v9q2\""
Jul 09 19:00:54.071633 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071601    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gft7c\" (UniqueName: \"kubernetes.io/projected/f9bd1049-aec4-4152-b132-4ea86a21dd7e-kube-api-access-gft7c\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.071799 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071640    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.071799 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071668    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-wtmp\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.071922 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071798    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-tls\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.071922 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071832    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9bd1049-aec4-4152-b132-4ea86a21dd7e-metrics-client-ca\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.071922 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071888    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-root\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.071922 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071913    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-textfile\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.072090 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071947    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.072090 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.071981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-sys\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173329 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173299    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-sys\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173475 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173336    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gft7c\" (UniqueName: \"kubernetes.io/projected/f9bd1049-aec4-4152-b132-4ea86a21dd7e-kube-api-access-gft7c\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173475 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173373    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173475 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173411    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-sys\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173645 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173500    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-wtmp\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173645 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173592    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-tls\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173645 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173614    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9bd1049-aec4-4152-b132-4ea86a21dd7e-metrics-client-ca\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173811 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173654    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-root\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173811 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-root\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173915 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173863    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-textfile\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.173915 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.173902    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.174142 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.174120    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-wtmp\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.174142 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.174136    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-textfile\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.174295 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.174278    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9bd1049-aec4-4152-b132-4ea86a21dd7e-metrics-client-ca\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.174416 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.174398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-accelerators-collector-config\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.176446 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.176428    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.176558 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.176531    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f9bd1049-aec4-4152-b132-4ea86a21dd7e-node-exporter-tls\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.183955 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.183925    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gft7c\" (UniqueName: \"kubernetes.io/projected/f9bd1049-aec4-4152-b132-4ea86a21dd7e-kube-api-access-gft7c\") pod \"node-exporter-n6qfs\" (UID: \"f9bd1049-aec4-4152-b132-4ea86a21dd7e\") " pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.341782 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:54.341706    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-n6qfs"
Jul 09 19:00:54.386120 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:00:54.386094    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9bd1049_aec4_4152_b132_4ea86a21dd7e.slice/crio-4578a9f63f0a6856ee5f118770a04638da796cbc46fe8fe26c054e356cb22578 WatchSource:0}: Error finding container 4578a9f63f0a6856ee5f118770a04638da796cbc46fe8fe26c054e356cb22578: Status 404 returned error can't find the container with id 4578a9f63f0a6856ee5f118770a04638da796cbc46fe8fe26c054e356cb22578
Jul 09 19:00:55.341299 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:55.341264    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n6qfs" event={"ID":"f9bd1049-aec4-4152-b132-4ea86a21dd7e","Type":"ContainerStarted","Data":"4578a9f63f0a6856ee5f118770a04638da796cbc46fe8fe26c054e356cb22578"}
Jul 09 19:00:56.202732 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:56.202703    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-2r4wh"
Jul 09 19:00:57.349955 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:57.349874    2569 generic.go:356] "Generic (PLEG): container finished" podID="f9bd1049-aec4-4152-b132-4ea86a21dd7e" containerID="6c3f094043826dffa0c2b2cd7d2ee3e8b0c39287599ea981e5a00c7924e92e8b" exitCode=0
Jul 09 19:00:57.350271 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:57.349958    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n6qfs" event={"ID":"f9bd1049-aec4-4152-b132-4ea86a21dd7e","Type":"ContainerDied","Data":"6c3f094043826dffa0c2b2cd7d2ee3e8b0c39287599ea981e5a00c7924e92e8b"}
Jul 09 19:00:58.355091 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:58.355055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n6qfs" event={"ID":"f9bd1049-aec4-4152-b132-4ea86a21dd7e","Type":"ContainerStarted","Data":"dc025fd919c15057884f060b38f347258bf4b8b7198ddff54ca7543348beb03e"}
Jul 09 19:00:58.355644 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:58.355151    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-n6qfs" event={"ID":"f9bd1049-aec4-4152-b132-4ea86a21dd7e","Type":"ContainerStarted","Data":"be38a36e33c893b647ad68de6375ea949dc4076a0b5adec0e5bb21a8312cf7b4"}
Jul 09 19:00:58.376911 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:00:58.376860    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-n6qfs" podStartSLOduration=2.646361901 podStartE2EDuration="5.376846692s" totalImagesPullingTime="2.622743266s" totalInitContainerRuntime="107.741525ms" isStatefulPod=true podCreationTimestamp="2026-07-09 19:00:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:00:58.375159028 +0000 UTC m=+100.958591243" watchObservedRunningTime="2026-07-09 19:00:58.376846692 +0000 UTC m=+100.960278873"
Jul 09 19:01:32.477387 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:01:32.477357    2569 generic.go:356] "Generic (PLEG): container finished" podID="d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b" containerID="17cd146388bc1446c952819ce7f498e6beaf83d4324ce1e9e01b2a3bc0a138a8" exitCode=0
Jul 09 19:01:32.477865 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:01:32.477402    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh" event={"ID":"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b","Type":"ContainerDied","Data":"17cd146388bc1446c952819ce7f498e6beaf83d4324ce1e9e01b2a3bc0a138a8"}
Jul 09 19:01:32.477865 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:01:32.477766    2569 scope.go:117] "RemoveContainer" containerID="17cd146388bc1446c952819ce7f498e6beaf83d4324ce1e9e01b2a3bc0a138a8"
Jul 09 19:01:33.483464 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:01:33.483431    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-q6nxh" event={"ID":"d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b","Type":"ContainerStarted","Data":"d8aee88a791759e3458dac91e23e197a22844141b717e96f25f7e1afb108f04e"}
Jul 09 19:03:04.305033 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.304996    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"]
Jul 09 19:03:04.307460 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.307443    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.310731 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.310706    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 19:03:04.310831 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.310706    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 19:03:04.312041 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.312024    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h9prp\""
Jul 09 19:03:04.317021 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.317000    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"]
Jul 09 19:03:04.428020 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.427990    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzgr7\" (UniqueName: \"kubernetes.io/projected/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-kube-api-access-bzgr7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.428175 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.428037    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.428175 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.428102    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.528409 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.528378    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.528549 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.528415    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzgr7\" (UniqueName: \"kubernetes.io/projected/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-kube-api-access-bzgr7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.528549 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.528534    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.528782 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.528766    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.528827 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.528814    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.537125 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.537103    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzgr7\" (UniqueName: \"kubernetes.io/projected/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-kube-api-access-bzgr7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.616610 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.616589    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:04.744409 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.744240    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"]
Jul 09 19:03:04.747042 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:03:04.747018    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ffeded5_f8ee_4a49_9bf1_a45135d5d7bd.slice/crio-d5f9ab6b217714e5b4d1622db5a976e98bd146d18d318e7cb077bb40f5ea1501 WatchSource:0}: Error finding container d5f9ab6b217714e5b4d1622db5a976e98bd146d18d318e7cb077bb40f5ea1501: Status 404 returned error can't find the container with id d5f9ab6b217714e5b4d1622db5a976e98bd146d18d318e7cb077bb40f5ea1501
Jul 09 19:03:04.809365 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:04.809330    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj" event={"ID":"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd","Type":"ContainerStarted","Data":"d5f9ab6b217714e5b4d1622db5a976e98bd146d18d318e7cb077bb40f5ea1501"}
Jul 09 19:03:14.848348 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:14.848252    2569 generic.go:356] "Generic (PLEG): container finished" podID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerID="37327cce2e7df23188d0fcff6061f54f62558c9f8ee4d3ee56838e27134cbf4a" exitCode=0
Jul 09 19:03:14.848348 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:14.848295    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj" event={"ID":"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd","Type":"ContainerDied","Data":"37327cce2e7df23188d0fcff6061f54f62558c9f8ee4d3ee56838e27134cbf4a"}
Jul 09 19:03:17.862077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:17.862044    2569 generic.go:356] "Generic (PLEG): container finished" podID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerID="a71111e18db666b1247e8dd7c6af838e9483e55404079f4b0363eb476d666c1f" exitCode=0
Jul 09 19:03:17.862409 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:17.862106    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj" event={"ID":"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd","Type":"ContainerDied","Data":"a71111e18db666b1247e8dd7c6af838e9483e55404079f4b0363eb476d666c1f"}
Jul 09 19:03:26.899728 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:26.899670    2569 generic.go:356] "Generic (PLEG): container finished" podID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerID="5e951052ca478d10882f4101ff78c31d9826134422a8f1a1e7d570d50632e87a" exitCode=0
Jul 09 19:03:26.900123 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:26.899748    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj" event={"ID":"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd","Type":"ContainerDied","Data":"5e951052ca478d10882f4101ff78c31d9826134422a8f1a1e7d570d50632e87a"}
Jul 09 19:03:28.030357 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.030331    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:28.127732 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.127702    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-util\") pod \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") "
Jul 09 19:03:28.127896 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.127791    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzgr7\" (UniqueName: \"kubernetes.io/projected/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-kube-api-access-bzgr7\") pod \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") "
Jul 09 19:03:28.127896 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.127823    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-bundle\") pod \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\" (UID: \"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd\") "
Jul 09 19:03:28.128461 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.128434    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-bundle" (OuterVolumeSpecName: "bundle") pod "0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" (UID: "0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:28.129992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.129969    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-kube-api-access-bzgr7" (OuterVolumeSpecName: "kube-api-access-bzgr7") pod "0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" (UID: "0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd"). InnerVolumeSpecName "kube-api-access-bzgr7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:03:28.132571 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.132548    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-util" (OuterVolumeSpecName: "util") pod "0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" (UID: "0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:28.229242 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.229162    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-util\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:03:28.229242 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.229189    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzgr7\" (UniqueName: \"kubernetes.io/projected/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-kube-api-access-bzgr7\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:03:28.229242 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.229199    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd-bundle\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:03:28.908040 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.908017    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj"
Jul 09 19:03:28.908231 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.908016    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540g6cqj" event={"ID":"0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd","Type":"ContainerDied","Data":"d5f9ab6b217714e5b4d1622db5a976e98bd146d18d318e7cb077bb40f5ea1501"}
Jul 09 19:03:28.908231 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:28.908134    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5f9ab6b217714e5b4d1622db5a976e98bd146d18d318e7cb077bb40f5ea1501"
Jul 09 19:03:32.334172 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334139    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"]
Jul 09 19:03:32.334601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334387    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerName="pull"
Jul 09 19:03:32.334601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334397    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerName="pull"
Jul 09 19:03:32.334601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334425    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerName="util"
Jul 09 19:03:32.334601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334431    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerName="util"
Jul 09 19:03:32.334601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334437    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerName="extract"
Jul 09 19:03:32.334601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334442    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerName="extract"
Jul 09 19:03:32.334601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.334492    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ffeded5-f8ee-4a49-9bf1-a45135d5d7bd" containerName="extract"
Jul 09 19:03:32.352126 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.352102    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"]
Jul 09 19:03:32.352243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.352199    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.355426 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.355393    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 09 19:03:32.355559 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.355449    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-rtp5r\""
Jul 09 19:03:32.355829 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.355811    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 09 19:03:32.460626 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.460586    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986a3107-0f5d-4a83-b1a6-6f3356d1c207-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7wsr5\" (UID: \"986a3107-0f5d-4a83-b1a6-6f3356d1c207\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.460626 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.460627    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbh52\" (UniqueName: \"kubernetes.io/projected/986a3107-0f5d-4a83-b1a6-6f3356d1c207-kube-api-access-kbh52\") pod \"cert-manager-operator-controller-manager-55c444bf86-7wsr5\" (UID: \"986a3107-0f5d-4a83-b1a6-6f3356d1c207\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.561751 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.561718    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986a3107-0f5d-4a83-b1a6-6f3356d1c207-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7wsr5\" (UID: \"986a3107-0f5d-4a83-b1a6-6f3356d1c207\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.561751 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.561756    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbh52\" (UniqueName: \"kubernetes.io/projected/986a3107-0f5d-4a83-b1a6-6f3356d1c207-kube-api-access-kbh52\") pod \"cert-manager-operator-controller-manager-55c444bf86-7wsr5\" (UID: \"986a3107-0f5d-4a83-b1a6-6f3356d1c207\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.562146 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.562122    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/986a3107-0f5d-4a83-b1a6-6f3356d1c207-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7wsr5\" (UID: \"986a3107-0f5d-4a83-b1a6-6f3356d1c207\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.570720 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.570677    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbh52\" (UniqueName: \"kubernetes.io/projected/986a3107-0f5d-4a83-b1a6-6f3356d1c207-kube-api-access-kbh52\") pod \"cert-manager-operator-controller-manager-55c444bf86-7wsr5\" (UID: \"986a3107-0f5d-4a83-b1a6-6f3356d1c207\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.661804 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.661777    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"
Jul 09 19:03:32.822970 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.822946    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5"]
Jul 09 19:03:32.825080 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:03:32.825050    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod986a3107_0f5d_4a83_b1a6_6f3356d1c207.slice/crio-13dd9989a094714ef84d9761ee5ac83c078bc681e50fd3da36b1cfa50ebb5555 WatchSource:0}: Error finding container 13dd9989a094714ef84d9761ee5ac83c078bc681e50fd3da36b1cfa50ebb5555: Status 404 returned error can't find the container with id 13dd9989a094714ef84d9761ee5ac83c078bc681e50fd3da36b1cfa50ebb5555
Jul 09 19:03:32.923827 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:32.923745    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5" event={"ID":"986a3107-0f5d-4a83-b1a6-6f3356d1c207","Type":"ContainerStarted","Data":"13dd9989a094714ef84d9761ee5ac83c078bc681e50fd3da36b1cfa50ebb5555"}
Jul 09 19:03:35.938045 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:35.938009    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5" event={"ID":"986a3107-0f5d-4a83-b1a6-6f3356d1c207","Type":"ContainerStarted","Data":"7c4fa74cbac0d2811fb6be9fd7d2acf7aa1c96e5a6304d4d80532208f0f1b8de"}
Jul 09 19:03:35.961172 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:35.961121    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7wsr5" podStartSLOduration=1.954705508 podStartE2EDuration="3.961106137s" totalImagesPullingTime="2.006400629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:35.958492983 +0000 UTC m=+258.541925181" watchObservedRunningTime="2026-07-09 19:03:35.961106137 +0000 UTC m=+258.544538316"
Jul 09 19:03:36.908119 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:36.908084    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"]
Jul 09 19:03:36.910387 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:36.910364    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:36.913434 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:36.913414    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h9prp\""
Jul 09 19:03:36.913552 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:36.913448    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 19:03:36.914820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:36.914804    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 19:03:36.921503 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:36.921483    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"]
Jul 09 19:03:37.102779 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.102742    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.103162 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.102789    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94sfs\" (UniqueName: \"kubernetes.io/projected/f7c01380-b1b5-43da-8aa7-1d483302f38a-kube-api-access-94sfs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.103162 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.102853    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.204051 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.203956    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.204051 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.204004    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94sfs\" (UniqueName: \"kubernetes.io/projected/f7c01380-b1b5-43da-8aa7-1d483302f38a-kube-api-access-94sfs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.204051 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.204035    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.204331 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.204297    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.204417 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.204397    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.213448 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.213425    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94sfs\" (UniqueName: \"kubernetes.io/projected/f7c01380-b1b5-43da-8aa7-1d483302f38a-kube-api-access-94sfs\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.219283 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.219266    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:37.348280 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.348258    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"]
Jul 09 19:03:37.350341 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:03:37.350305    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7c01380_b1b5_43da_8aa7_1d483302f38a.slice/crio-e1ef14e448c64f57e37dd1c61ed227e4d9482ebba143571845f2e37b9742deba WatchSource:0}: Error finding container e1ef14e448c64f57e37dd1c61ed227e4d9482ebba143571845f2e37b9742deba: Status 404 returned error can't find the container with id e1ef14e448c64f57e37dd1c61ed227e4d9482ebba143571845f2e37b9742deba
Jul 09 19:03:37.946235 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.946199    2569 generic.go:356] "Generic (PLEG): container finished" podID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerID="16ab3c64df496541111936b8bc8edb19fa8c95fa0604ac48dd8e59b4d0292a6f" exitCode=0
Jul 09 19:03:37.946420 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.946278    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98" event={"ID":"f7c01380-b1b5-43da-8aa7-1d483302f38a","Type":"ContainerDied","Data":"16ab3c64df496541111936b8bc8edb19fa8c95fa0604ac48dd8e59b4d0292a6f"}
Jul 09 19:03:37.946420 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:37.946333    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98" event={"ID":"f7c01380-b1b5-43da-8aa7-1d483302f38a","Type":"ContainerStarted","Data":"e1ef14e448c64f57e37dd1c61ed227e4d9482ebba143571845f2e37b9742deba"}
Jul 09 19:03:39.404370 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.404336    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"]
Jul 09 19:03:39.409244 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.407891    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.411183 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.411160    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wvgv2\""
Jul 09 19:03:39.412661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.412589    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 09 19:03:39.412661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.412612    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 09 19:03:39.415162 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.415145    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"]
Jul 09 19:03:39.423254 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.423233    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a38b5e37-3945-43e2-9808-f595284521e5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xd4dx\" (UID: \"a38b5e37-3945-43e2-9808-f595284521e5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.423341 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.423261    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmsnr\" (UniqueName: \"kubernetes.io/projected/a38b5e37-3945-43e2-9808-f595284521e5-kube-api-access-nmsnr\") pod \"cert-manager-cainjector-559cdd8c87-xd4dx\" (UID: \"a38b5e37-3945-43e2-9808-f595284521e5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.524380 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.524347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a38b5e37-3945-43e2-9808-f595284521e5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xd4dx\" (UID: \"a38b5e37-3945-43e2-9808-f595284521e5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.524531 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.524395    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmsnr\" (UniqueName: \"kubernetes.io/projected/a38b5e37-3945-43e2-9808-f595284521e5-kube-api-access-nmsnr\") pod \"cert-manager-cainjector-559cdd8c87-xd4dx\" (UID: \"a38b5e37-3945-43e2-9808-f595284521e5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.533447 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.533418    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a38b5e37-3945-43e2-9808-f595284521e5-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-xd4dx\" (UID: \"a38b5e37-3945-43e2-9808-f595284521e5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.533543 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.533488    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmsnr\" (UniqueName: \"kubernetes.io/projected/a38b5e37-3945-43e2-9808-f595284521e5-kube-api-access-nmsnr\") pod \"cert-manager-cainjector-559cdd8c87-xd4dx\" (UID: \"a38b5e37-3945-43e2-9808-f595284521e5\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.727915 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.727837    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"
Jul 09 19:03:39.875779 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:03:39.875743    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda38b5e37_3945_43e2_9808_f595284521e5.slice/crio-b568bfb44e810761292dbb21b11ce46f158976a2b535b3ab32c96346083f3f04 WatchSource:0}: Error finding container b568bfb44e810761292dbb21b11ce46f158976a2b535b3ab32c96346083f3f04: Status 404 returned error can't find the container with id b568bfb44e810761292dbb21b11ce46f158976a2b535b3ab32c96346083f3f04
Jul 09 19:03:39.877579 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.877510    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx"]
Jul 09 19:03:39.955670 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:39.955638    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx" event={"ID":"a38b5e37-3945-43e2-9808-f595284521e5","Type":"ContainerStarted","Data":"b568bfb44e810761292dbb21b11ce46f158976a2b535b3ab32c96346083f3f04"}
Jul 09 19:03:40.963017 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:40.962978    2569 generic.go:356] "Generic (PLEG): container finished" podID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerID="fd58f036c616f9a8487451380ccdf9b86d58f5baddeaba1e4be0013614faea90" exitCode=0
Jul 09 19:03:40.963446 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:40.963016    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98" event={"ID":"f7c01380-b1b5-43da-8aa7-1d483302f38a","Type":"ContainerDied","Data":"fd58f036c616f9a8487451380ccdf9b86d58f5baddeaba1e4be0013614faea90"}
Jul 09 19:03:41.970701 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:41.970634    2569 generic.go:356] "Generic (PLEG): container finished" podID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerID="1a5310a6029ee963736fb2ef66101eb2c9a53b436c1b4423680783a2ee3af766" exitCode=0
Jul 09 19:03:41.971087 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:41.970724    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98" event={"ID":"f7c01380-b1b5-43da-8aa7-1d483302f38a","Type":"ContainerDied","Data":"1a5310a6029ee963736fb2ef66101eb2c9a53b436c1b4423680783a2ee3af766"}
Jul 09 19:03:42.976757 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:42.976723    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx" event={"ID":"a38b5e37-3945-43e2-9808-f595284521e5","Type":"ContainerStarted","Data":"3bd51c7f93638cbf50b90ec008470b00ec7fd03f56502d747eb03d2894c9c936"}
Jul 09 19:03:42.993013 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:42.992960    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-xd4dx" podStartSLOduration=1.333260638 podStartE2EDuration="3.992938438s" totalImagesPullingTime="2.6596778s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:42.992050338 +0000 UTC m=+265.575482518" watchObservedRunningTime="2026-07-09 19:03:42.992938438 +0000 UTC m=+265.576370620"
Jul 09 19:03:43.113805 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.113783    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:43.154074 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.154044    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-94sfs\" (UniqueName: \"kubernetes.io/projected/f7c01380-b1b5-43da-8aa7-1d483302f38a-kube-api-access-94sfs\") pod \"f7c01380-b1b5-43da-8aa7-1d483302f38a\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") "
Jul 09 19:03:43.154196 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.154080    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-util\") pod \"f7c01380-b1b5-43da-8aa7-1d483302f38a\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") "
Jul 09 19:03:43.154196 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.154105    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-bundle\") pod \"f7c01380-b1b5-43da-8aa7-1d483302f38a\" (UID: \"f7c01380-b1b5-43da-8aa7-1d483302f38a\") "
Jul 09 19:03:43.154490 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.154465    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-bundle" (OuterVolumeSpecName: "bundle") pod "f7c01380-b1b5-43da-8aa7-1d483302f38a" (UID: "f7c01380-b1b5-43da-8aa7-1d483302f38a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:43.156239 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.156215    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7c01380-b1b5-43da-8aa7-1d483302f38a-kube-api-access-94sfs" (OuterVolumeSpecName: "kube-api-access-94sfs") pod "f7c01380-b1b5-43da-8aa7-1d483302f38a" (UID: "f7c01380-b1b5-43da-8aa7-1d483302f38a"). InnerVolumeSpecName "kube-api-access-94sfs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:03:43.160144 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.160113    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-util" (OuterVolumeSpecName: "util") pod "f7c01380-b1b5-43da-8aa7-1d483302f38a" (UID: "f7c01380-b1b5-43da-8aa7-1d483302f38a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:03:43.255000 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.254911    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-94sfs\" (UniqueName: \"kubernetes.io/projected/f7c01380-b1b5-43da-8aa7-1d483302f38a-kube-api-access-94sfs\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:03:43.255000 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.254941    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-util\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:03:43.255000 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.254952    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f7c01380-b1b5-43da-8aa7-1d483302f38a-bundle\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:03:43.983265 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.983241    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98"
Jul 09 19:03:43.983737 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.983268    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fthc98" event={"ID":"f7c01380-b1b5-43da-8aa7-1d483302f38a","Type":"ContainerDied","Data":"e1ef14e448c64f57e37dd1c61ed227e4d9482ebba143571845f2e37b9742deba"}
Jul 09 19:03:43.983737 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:43.983327    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e1ef14e448c64f57e37dd1c61ed227e4d9482ebba143571845f2e37b9742deba"
Jul 09 19:03:54.600390 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600359    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"]
Jul 09 19:03:54.600807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600611    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerName="pull"
Jul 09 19:03:54.600807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600621    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerName="pull"
Jul 09 19:03:54.600807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600637    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerName="extract"
Jul 09 19:03:54.600807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600643    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerName="extract"
Jul 09 19:03:54.600807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600651    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerName="util"
Jul 09 19:03:54.600807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600657    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerName="util"
Jul 09 19:03:54.600807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.600719    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7c01380-b1b5-43da-8aa7-1d483302f38a" containerName="extract"
Jul 09 19:03:54.602986 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.602967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:54.606278 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.606257    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 09 19:03:54.606278 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.606277    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 09 19:03:54.607636 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.607623    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-zhpmw\""
Jul 09 19:03:54.610789 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.610768    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"]
Jul 09 19:03:54.636644 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.636613    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlnxc\" (UniqueName: \"kubernetes.io/projected/821de831-177e-4081-9484-aa4710829b1f-kube-api-access-dlnxc\") pod \"openshift-lws-operator-fd8ccff4c-kvckn\" (UID: \"821de831-177e-4081-9484-aa4710829b1f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:54.636791 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.636650    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/821de831-177e-4081-9484-aa4710829b1f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kvckn\" (UID: \"821de831-177e-4081-9484-aa4710829b1f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:54.737143 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.737111    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlnxc\" (UniqueName: \"kubernetes.io/projected/821de831-177e-4081-9484-aa4710829b1f-kube-api-access-dlnxc\") pod \"openshift-lws-operator-fd8ccff4c-kvckn\" (UID: \"821de831-177e-4081-9484-aa4710829b1f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:54.737291 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.737146    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/821de831-177e-4081-9484-aa4710829b1f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kvckn\" (UID: \"821de831-177e-4081-9484-aa4710829b1f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:54.737517 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.737501    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/821de831-177e-4081-9484-aa4710829b1f-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kvckn\" (UID: \"821de831-177e-4081-9484-aa4710829b1f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:54.746497 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.746474    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlnxc\" (UniqueName: \"kubernetes.io/projected/821de831-177e-4081-9484-aa4710829b1f-kube-api-access-dlnxc\") pod \"openshift-lws-operator-fd8ccff4c-kvckn\" (UID: \"821de831-177e-4081-9484-aa4710829b1f\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:54.912306 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:54.912271    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"
Jul 09 19:03:55.041402 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:55.041381    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn"]
Jul 09 19:03:55.043290 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:03:55.043263    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod821de831_177e_4081_9484_aa4710829b1f.slice/crio-4303cfe79b575528a08d6ff4f565322a92ce4612022e254a11c6355708743d1d WatchSource:0}: Error finding container 4303cfe79b575528a08d6ff4f565322a92ce4612022e254a11c6355708743d1d: Status 404 returned error can't find the container with id 4303cfe79b575528a08d6ff4f565322a92ce4612022e254a11c6355708743d1d
Jul 09 19:03:56.030331 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:56.030299    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn" event={"ID":"821de831-177e-4081-9484-aa4710829b1f","Type":"ContainerStarted","Data":"4303cfe79b575528a08d6ff4f565322a92ce4612022e254a11c6355708743d1d"}
Jul 09 19:03:58.040088 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:58.040055    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn" event={"ID":"821de831-177e-4081-9484-aa4710829b1f","Type":"ContainerStarted","Data":"9ef352bf996c4cae52347fc8cb920dff22495d1dd8649805ad2534d591ec815b"}
Jul 09 19:03:58.056894 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:03:58.056849    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kvckn" podStartSLOduration=1.703139094 podStartE2EDuration="4.056835809s" totalImagesPullingTime="2.353696715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:03:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:03:58.056148327 +0000 UTC m=+280.639580526" watchObservedRunningTime="2026-07-09 19:03:58.056835809 +0000 UTC m=+280.640267988"
Jul 09 19:04:02.450973 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.450937    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"]
Jul 09 19:04:02.453533 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.453517    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.456329 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.456308    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 09 19:04:02.456434 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.456307    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 09 19:04:02.457545 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.457527    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-h9prp\""
Jul 09 19:04:02.461987 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.461962    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"]
Jul 09 19:04:02.495441 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.495410    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzv8x\" (UniqueName: \"kubernetes.io/projected/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-kube-api-access-bzv8x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.495550 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.495447    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.495550 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.495479    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.596610 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.596574    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.596788 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.596628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzv8x\" (UniqueName: \"kubernetes.io/projected/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-kube-api-access-bzv8x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.596788 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.596661    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.596953 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.596934    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.597009 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.596993    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.605550 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.605532    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzv8x\" (UniqueName: \"kubernetes.io/projected/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-kube-api-access-bzv8x\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.763600 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.763518    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:02.896130 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:02.896108    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"]
Jul 09 19:04:02.898091 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:02.898061    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda899e1c8_035d_4f07_8ecd_5a9b21c3de7a.slice/crio-7e01c41dbfceed3f82848495f490fbfaba0ce88b676a8a4050df55b1b894735b WatchSource:0}: Error finding container 7e01c41dbfceed3f82848495f490fbfaba0ce88b676a8a4050df55b1b894735b: Status 404 returned error can't find the container with id 7e01c41dbfceed3f82848495f490fbfaba0ce88b676a8a4050df55b1b894735b
Jul 09 19:04:03.060677 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:03.060593    2569 generic.go:356] "Generic (PLEG): container finished" podID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerID="ea2031221a69864ce560e68b0e75fd3fe4bb990d94c197d72d37a53216e2526b" exitCode=0
Jul 09 19:04:03.060677 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:03.060637    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l" event={"ID":"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a","Type":"ContainerDied","Data":"ea2031221a69864ce560e68b0e75fd3fe4bb990d94c197d72d37a53216e2526b"}
Jul 09 19:04:03.060910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:03.060703    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l" event={"ID":"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a","Type":"ContainerStarted","Data":"7e01c41dbfceed3f82848495f490fbfaba0ce88b676a8a4050df55b1b894735b"}
Jul 09 19:04:04.066339 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:04.066300    2569 generic.go:356] "Generic (PLEG): container finished" podID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerID="986afa95a4011b36d746af8568bb6e8f27428635f863d9a94f51590fdf286c41" exitCode=0
Jul 09 19:04:04.066869 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:04.066381    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l" event={"ID":"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a","Type":"ContainerDied","Data":"986afa95a4011b36d746af8568bb6e8f27428635f863d9a94f51590fdf286c41"}
Jul 09 19:04:05.074870 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:05.074840    2569 generic.go:356] "Generic (PLEG): container finished" podID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerID="3469aa62616967818fd58c05a2390716c4a1fd4716755ec1bda1a9c5bbd2ca7d" exitCode=0
Jul 09 19:04:05.075263 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:05.074924    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l" event={"ID":"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a","Type":"ContainerDied","Data":"3469aa62616967818fd58c05a2390716c4a1fd4716755ec1bda1a9c5bbd2ca7d"}
Jul 09 19:04:06.201981 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.201957    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:06.326308 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.326279    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-bundle\") pod \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") "
Jul 09 19:04:06.326463 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.326373    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzv8x\" (UniqueName: \"kubernetes.io/projected/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-kube-api-access-bzv8x\") pod \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") "
Jul 09 19:04:06.326463 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.326395    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-util\") pod \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\" (UID: \"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a\") "
Jul 09 19:04:06.327129 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.327034    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-bundle" (OuterVolumeSpecName: "bundle") pod "a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" (UID: "a899e1c8-035d-4f07-8ecd-5a9b21c3de7a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:04:06.328587 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.328558    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-kube-api-access-bzv8x" (OuterVolumeSpecName: "kube-api-access-bzv8x") pod "a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" (UID: "a899e1c8-035d-4f07-8ecd-5a9b21c3de7a"). InnerVolumeSpecName "kube-api-access-bzv8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:04:06.332408 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.332354    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-util" (OuterVolumeSpecName: "util") pod "a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" (UID: "a899e1c8-035d-4f07-8ecd-5a9b21c3de7a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:04:06.427082 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.427045    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzv8x\" (UniqueName: \"kubernetes.io/projected/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-kube-api-access-bzv8x\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:06.427082 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.427077    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-util\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:06.427082 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:06.427087    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a899e1c8-035d-4f07-8ecd-5a9b21c3de7a-bundle\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:07.084802 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:07.084773    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l"
Jul 09 19:04:07.085002 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:07.084770    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afd884l" event={"ID":"a899e1c8-035d-4f07-8ecd-5a9b21c3de7a","Type":"ContainerDied","Data":"7e01c41dbfceed3f82848495f490fbfaba0ce88b676a8a4050df55b1b894735b"}
Jul 09 19:04:07.085002 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:07.084885    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e01c41dbfceed3f82848495f490fbfaba0ce88b676a8a4050df55b1b894735b"
Jul 09 19:04:14.333425 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333391    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"]
Jul 09 19:04:14.333820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333656    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerName="pull"
Jul 09 19:04:14.333820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333667    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerName="pull"
Jul 09 19:04:14.333820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333702    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerName="util"
Jul 09 19:04:14.333820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333707    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerName="util"
Jul 09 19:04:14.333820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333715    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerName="extract"
Jul 09 19:04:14.333820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333721    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerName="extract"
Jul 09 19:04:14.333820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.333771    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="a899e1c8-035d-4f07-8ecd-5a9b21c3de7a" containerName="extract"
Jul 09 19:04:14.335627 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.335612    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.338790 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.338764    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 09 19:04:14.338935 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.338791    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 09 19:04:14.338935 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.338911    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 09 19:04:14.339521 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.339503    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 09 19:04:14.344981 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.344959    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-67swh\""
Jul 09 19:04:14.356665 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.356638    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"]
Jul 09 19:04:14.385975 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.385953    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c82727eb-817e-4415-9b10-9b8b0d6c8803-apiservice-cert\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.386115 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.386012    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c82727eb-817e-4415-9b10-9b8b0d6c8803-manifests\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.386115 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.386041    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn79f\" (UniqueName: \"kubernetes.io/projected/c82727eb-817e-4415-9b10-9b8b0d6c8803-kube-api-access-kn79f\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.386115 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.386077    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c82727eb-817e-4415-9b10-9b8b0d6c8803-tmp\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.386115 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.386093    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c82727eb-817e-4415-9b10-9b8b0d6c8803-webhook-cert\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.487206 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.487171    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c82727eb-817e-4415-9b10-9b8b0d6c8803-tmp\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.487206 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.487207    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c82727eb-817e-4415-9b10-9b8b0d6c8803-webhook-cert\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.487429 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.487240    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c82727eb-817e-4415-9b10-9b8b0d6c8803-apiservice-cert\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.487429 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.487315    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c82727eb-817e-4415-9b10-9b8b0d6c8803-manifests\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.487532 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.487439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn79f\" (UniqueName: \"kubernetes.io/projected/c82727eb-817e-4415-9b10-9b8b0d6c8803-kube-api-access-kn79f\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.487697 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.487667    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c82727eb-817e-4415-9b10-9b8b0d6c8803-manifests\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.489727 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.489700    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c82727eb-817e-4415-9b10-9b8b0d6c8803-tmp\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.489876 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.489852    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c82727eb-817e-4415-9b10-9b8b0d6c8803-webhook-cert\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.490014 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.489997    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c82727eb-817e-4415-9b10-9b8b0d6c8803-apiservice-cert\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.504838 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.504817    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn79f\" (UniqueName: \"kubernetes.io/projected/c82727eb-817e-4415-9b10-9b8b0d6c8803-kube-api-access-kn79f\") pod \"opendatahub-operator-controller-manager-779db88bb-7fl8f\" (UID: \"c82727eb-817e-4415-9b10-9b8b0d6c8803\") " pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.646271 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.646235    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:14.834469 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:14.834446    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"]
Jul 09 19:04:14.837259 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:14.837232    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc82727eb_817e_4415_9b10_9b8b0d6c8803.slice/crio-5f296c5145d01bf7c26b5f92b5d7075450b23ce698f4e13f04f0f47afe2e3bcc WatchSource:0}: Error finding container 5f296c5145d01bf7c26b5f92b5d7075450b23ce698f4e13f04f0f47afe2e3bcc: Status 404 returned error can't find the container with id 5f296c5145d01bf7c26b5f92b5d7075450b23ce698f4e13f04f0f47afe2e3bcc
Jul 09 19:04:15.119233 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:15.119202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f" event={"ID":"c82727eb-817e-4415-9b10-9b8b0d6c8803","Type":"ContainerStarted","Data":"5f296c5145d01bf7c26b5f92b5d7075450b23ce698f4e13f04f0f47afe2e3bcc"}
Jul 09 19:04:17.838476 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:17.838449    2569 kubelet.go:1648] "Image garbage collection succeeded"
Jul 09 19:04:18.135779 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:18.135743    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f" event={"ID":"c82727eb-817e-4415-9b10-9b8b0d6c8803","Type":"ContainerStarted","Data":"0e7f6b55b85f63e6c12cdee4bb68f7808931bd3cc052269f3479bfdfd49dbe44"}
Jul 09 19:04:19.141846 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:19.141813    2569 generic.go:356] "Generic (PLEG): container finished" podID="c82727eb-817e-4415-9b10-9b8b0d6c8803" containerID="0e7f6b55b85f63e6c12cdee4bb68f7808931bd3cc052269f3479bfdfd49dbe44" exitCode=0
Jul 09 19:04:19.142255 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:19.141858    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f" event={"ID":"c82727eb-817e-4415-9b10-9b8b0d6c8803","Type":"ContainerDied","Data":"0e7f6b55b85f63e6c12cdee4bb68f7808931bd3cc052269f3479bfdfd49dbe44"}
Jul 09 19:04:19.142457 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:19.142439    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 19:04:20.147414 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:20.147376    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f" event={"ID":"c82727eb-817e-4415-9b10-9b8b0d6c8803","Type":"ContainerStarted","Data":"2742b9c0716f3c60cb33b0daf1dc8e04302efa9674dd599d81e37cafbbcf8013"}
Jul 09 19:04:20.147826 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:20.147511    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:20.170106 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:20.170061    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f" podStartSLOduration=1.650833214 podStartE2EDuration="6.170045985s" totalImagesPullingTime="3.356983065s" totalInitContainerRuntime="1.162229706s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:14 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:20.169238605 +0000 UTC m=+302.752670785" watchObservedRunningTime="2026-07-09 19:04:20.170045985 +0000 UTC m=+302.753478164"
Jul 09 19:04:31.154214 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:31.154185    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-779db88bb-7fl8f"
Jul 09 19:04:39.668721 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.668675    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"]
Jul 09 19:04:39.671899 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.671881    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.675078 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.675055    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 09 19:04:39.675078 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.675075    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 09 19:04:39.675229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.675055    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-jkwf6\""
Jul 09 19:04:39.685983 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.685959    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"]
Jul 09 19:04:39.783629 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.783597    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lv5bf"]
Jul 09 19:04:39.786816 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.786786    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:39.789798 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.789773    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 09 19:04:39.789979 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.789961    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-f7clh\""
Jul 09 19:04:39.792547 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.792529    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4543215d-3a28-44f5-803d-014eb3ebbc9f-tls-certs\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.792639 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.792564    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4543215d-3a28-44f5-803d-014eb3ebbc9f-tmp\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.792639 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.792592    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xdxb\" (UniqueName: \"kubernetes.io/projected/4543215d-3a28-44f5-803d-014eb3ebbc9f-kube-api-access-2xdxb\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.802131 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.802110    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lv5bf"]
Jul 09 19:04:39.893497 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.893461    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-cert\") pod \"odh-model-controller-7998c67dc4-lv5bf\" (UID: \"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3\") " pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:39.893651 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.893533    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4543215d-3a28-44f5-803d-014eb3ebbc9f-tls-certs\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.893651 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.893563    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldwwk\" (UniqueName: \"kubernetes.io/projected/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-kube-api-access-ldwwk\") pod \"odh-model-controller-7998c67dc4-lv5bf\" (UID: \"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3\") " pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:39.893651 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.893591    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4543215d-3a28-44f5-803d-014eb3ebbc9f-tmp\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.893835 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.893650    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xdxb\" (UniqueName: \"kubernetes.io/projected/4543215d-3a28-44f5-803d-014eb3ebbc9f-kube-api-access-2xdxb\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.896083 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.896051    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4543215d-3a28-44f5-803d-014eb3ebbc9f-tmp\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.896372 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.896350    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4543215d-3a28-44f5-803d-014eb3ebbc9f-tls-certs\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.907752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.907726    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xdxb\" (UniqueName: \"kubernetes.io/projected/4543215d-3a28-44f5-803d-014eb3ebbc9f-kube-api-access-2xdxb\") pod \"kube-auth-proxy-7b6c4d4-jdvzz\" (UID: \"4543215d-3a28-44f5-803d-014eb3ebbc9f\") " pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.982061 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.981967    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"
Jul 09 19:04:39.994623 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.994590    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldwwk\" (UniqueName: \"kubernetes.io/projected/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-kube-api-access-ldwwk\") pod \"odh-model-controller-7998c67dc4-lv5bf\" (UID: \"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3\") " pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:39.994774 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:39.994656    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-cert\") pod \"odh-model-controller-7998c67dc4-lv5bf\" (UID: \"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3\") " pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:39.994845 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:39.994782    2569 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 09 19:04:39.994845 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:39.994841    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-cert podName:cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3 nodeName:}" failed. No retries permitted until 2026-07-09 19:04:40.494821445 +0000 UTC m=+323.078253605 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-cert") pod "odh-model-controller-7998c67dc4-lv5bf" (UID: "cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3") : secret "odh-model-controller-webhook-cert" not found
Jul 09 19:04:40.005701 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.005660    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldwwk\" (UniqueName: \"kubernetes.io/projected/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-kube-api-access-ldwwk\") pod \"odh-model-controller-7998c67dc4-lv5bf\" (UID: \"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3\") " pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:40.124649 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.124622    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz"]
Jul 09 19:04:40.127363 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:40.127337    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4543215d_3a28_44f5_803d_014eb3ebbc9f.slice/crio-39e52a18d02d25951fa20afd1f4470cf970813272dd32e7fedbfc892d73e1291 WatchSource:0}: Error finding container 39e52a18d02d25951fa20afd1f4470cf970813272dd32e7fedbfc892d73e1291: Status 404 returned error can't find the container with id 39e52a18d02d25951fa20afd1f4470cf970813272dd32e7fedbfc892d73e1291
Jul 09 19:04:40.235018 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.234917    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz" event={"ID":"4543215d-3a28-44f5-803d-014eb3ebbc9f","Type":"ContainerStarted","Data":"39e52a18d02d25951fa20afd1f4470cf970813272dd32e7fedbfc892d73e1291"}
Jul 09 19:04:40.272961 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.272926    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rbj5f"]
Jul 09 19:04:40.277040 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.277025    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:40.280273 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.280252    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 09 19:04:40.280273 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.280267    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-xtq65\""
Jul 09 19:04:40.291377 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.291349    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rbj5f"]
Jul 09 19:04:40.399098 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.399063    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c702c73-fae0-4b1f-8886-4577e75cc21f-tls-certs\") pod \"model-serving-api-54d89bd5fc-rbj5f\" (UID: \"2c702c73-fae0-4b1f-8886-4577e75cc21f\") " pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:40.399289 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.399141    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tprsl\" (UniqueName: \"kubernetes.io/projected/2c702c73-fae0-4b1f-8886-4577e75cc21f-kube-api-access-tprsl\") pod \"model-serving-api-54d89bd5fc-rbj5f\" (UID: \"2c702c73-fae0-4b1f-8886-4577e75cc21f\") " pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:40.500618 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.500509    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c702c73-fae0-4b1f-8886-4577e75cc21f-tls-certs\") pod \"model-serving-api-54d89bd5fc-rbj5f\" (UID: \"2c702c73-fae0-4b1f-8886-4577e75cc21f\") " pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:40.500618 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.500610    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-cert\") pod \"odh-model-controller-7998c67dc4-lv5bf\" (UID: \"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3\") " pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:40.500872 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.500648    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tprsl\" (UniqueName: \"kubernetes.io/projected/2c702c73-fae0-4b1f-8886-4577e75cc21f-kube-api-access-tprsl\") pod \"model-serving-api-54d89bd5fc-rbj5f\" (UID: \"2c702c73-fae0-4b1f-8886-4577e75cc21f\") " pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:40.500872 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:40.500663    2569 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 09 19:04:40.500872 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:40.500746    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c702c73-fae0-4b1f-8886-4577e75cc21f-tls-certs podName:2c702c73-fae0-4b1f-8886-4577e75cc21f nodeName:}" failed. No retries permitted until 2026-07-09 19:04:41.000731907 +0000 UTC m=+323.584164065 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2c702c73-fae0-4b1f-8886-4577e75cc21f-tls-certs") pod "model-serving-api-54d89bd5fc-rbj5f" (UID: "2c702c73-fae0-4b1f-8886-4577e75cc21f") : secret "model-serving-api-tls" not found
Jul 09 19:04:40.503196 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.503175    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3-cert\") pod \"odh-model-controller-7998c67dc4-lv5bf\" (UID: \"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3\") " pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:40.511864 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.511841    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tprsl\" (UniqueName: \"kubernetes.io/projected/2c702c73-fae0-4b1f-8886-4577e75cc21f-kube-api-access-tprsl\") pod \"model-serving-api-54d89bd5fc-rbj5f\" (UID: \"2c702c73-fae0-4b1f-8886-4577e75cc21f\") " pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:40.697308 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.697274    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:40.836788 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:40.836755    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-lv5bf"]
Jul 09 19:04:40.838498 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:40.838448    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf17df4d_4378_4cb8_ad20_6b4de3bf9ab3.slice/crio-f564d3875d3ddec4a02d5e29b0176a063c01089a59a59f721433d9cb9b65d81d WatchSource:0}: Error finding container f564d3875d3ddec4a02d5e29b0176a063c01089a59a59f721433d9cb9b65d81d: Status 404 returned error can't find the container with id f564d3875d3ddec4a02d5e29b0176a063c01089a59a59f721433d9cb9b65d81d
Jul 09 19:04:41.005205 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.005167    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c702c73-fae0-4b1f-8886-4577e75cc21f-tls-certs\") pod \"model-serving-api-54d89bd5fc-rbj5f\" (UID: \"2c702c73-fae0-4b1f-8886-4577e75cc21f\") " pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:41.008383 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.008354    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2c702c73-fae0-4b1f-8886-4577e75cc21f-tls-certs\") pod \"model-serving-api-54d89bd5fc-rbj5f\" (UID: \"2c702c73-fae0-4b1f-8886-4577e75cc21f\") " pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:41.155142 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.155108    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"]
Jul 09 19:04:41.158464 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.158432    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.161398 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.161368    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 09 19:04:41.161398 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.161386    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 09 19:04:41.161597 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.161449    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 09 19:04:41.161597 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.161391    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 09 19:04:41.161802 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.161777    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-zddjl\""
Jul 09 19:04:41.174548 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.174497    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"]
Jul 09 19:04:41.188340 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.188243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:41.243862 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.243776    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf" event={"ID":"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3","Type":"ContainerStarted","Data":"f564d3875d3ddec4a02d5e29b0176a063c01089a59a59f721433d9cb9b65d81d"}
Jul 09 19:04:41.308355 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.308318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6wxm\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-kube-api-access-k6wxm\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.308508 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.308371    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-cacerts\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.308508 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.308435    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.308508 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.308484    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-token\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.308608 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.308516    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-local-certs\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.308608 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.308550    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.308608 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.308600    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.355844 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.355814    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-rbj5f"]
Jul 09 19:04:41.358163 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:41.358132    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c702c73_fae0_4b1f_8886_4577e75cc21f.slice/crio-718ef3e59c585444ac4a5c72f70a7acf8caeca6266cbdda85f76982dad09e3a3 WatchSource:0}: Error finding container 718ef3e59c585444ac4a5c72f70a7acf8caeca6266cbdda85f76982dad09e3a3: Status 404 returned error can't find the container with id 718ef3e59c585444ac4a5c72f70a7acf8caeca6266cbdda85f76982dad09e3a3
Jul 09 19:04:41.409457 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.409368    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.409457 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.409416    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-token\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.409457 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.409439    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-local-certs\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.409796 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.409478    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.409796 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.409518    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.409796 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.409556    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6wxm\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-kube-api-access-k6wxm\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.409796 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.409582    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-cacerts\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.410470 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.410436    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.412544 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.412523    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.412646 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.412543    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-local-certs\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.412646 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.412603    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.412816 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.412795    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-cacerts\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.420618 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.420590    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6wxm\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-kube-api-access-k6wxm\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.420740 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.420640    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-token\") pod \"istiod-openshift-gateway-7574459656-rjt2x\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.471815 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.471789    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:41.618703 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:41.618651    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"]
Jul 09 19:04:41.621649 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:41.621612    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bb79f9_d75d_4b7b_8451_acd79b6eea2e.slice/crio-4d56f80ffa71d08432f205d06f083332840ad9d4a191f875d943ef78813ebf37 WatchSource:0}: Error finding container 4d56f80ffa71d08432f205d06f083332840ad9d4a191f875d943ef78813ebf37: Status 404 returned error can't find the container with id 4d56f80ffa71d08432f205d06f083332840ad9d4a191f875d943ef78813ebf37
Jul 09 19:04:42.253233 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:42.253188    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f" event={"ID":"2c702c73-fae0-4b1f-8886-4577e75cc21f","Type":"ContainerStarted","Data":"718ef3e59c585444ac4a5c72f70a7acf8caeca6266cbdda85f76982dad09e3a3"}
Jul 09 19:04:42.255075 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:42.255013    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x" event={"ID":"43bb79f9-d75d-4b7b-8451-acd79b6eea2e","Type":"ContainerStarted","Data":"4d56f80ffa71d08432f205d06f083332840ad9d4a191f875d943ef78813ebf37"}
Jul 09 19:04:46.506468 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:46.506434    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 19:04:46.506863 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:46.506495    2569 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"}
Jul 09 19:04:47.286659 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.286390    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz" event={"ID":"4543215d-3a28-44f5-803d-014eb3ebbc9f","Type":"ContainerStarted","Data":"35c5eee12177718ed0a4b738ee775df233b790945b016047807d8b327b6e4614"}
Jul 09 19:04:47.288400 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.288373    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f" event={"ID":"2c702c73-fae0-4b1f-8886-4577e75cc21f","Type":"ContainerStarted","Data":"c69965ef789c3e58be739f75e107c0bc75b4dc477310330126ac0c4003affae9"}
Jul 09 19:04:47.288845 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.288823    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:04:47.290922 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.290896    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x" event={"ID":"43bb79f9-d75d-4b7b-8451-acd79b6eea2e","Type":"ContainerStarted","Data":"c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b"}
Jul 09 19:04:47.291030 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.290969    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:47.293175 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.293116    2569 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-rjt2x container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 09 19:04:47.293258 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.293210    2569 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x" podUID="43bb79f9-d75d-4b7b-8451-acd79b6eea2e" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 09 19:04:47.293774 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.293659    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf" event={"ID":"cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3","Type":"ContainerStarted","Data":"4451e4ff44c8ee73e74bf6a39b1a07e53fe72e07f0921e963c5824179f19fcbd"}
Jul 09 19:04:47.293774 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.293741    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:47.307025 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.306986    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7b6c4d4-jdvzz" podStartSLOduration=1.93348454 podStartE2EDuration="8.306969059s" totalImagesPullingTime="6.373484519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:47.305344373 +0000 UTC m=+329.888776573" watchObservedRunningTime="2026-07-09 19:04:47.306969059 +0000 UTC m=+329.890401239"
Jul 09 19:04:47.323624 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.323581    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f" podStartSLOduration=2.180888491 podStartE2EDuration="7.323566089s" totalImagesPullingTime="5.142677598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:47.322886432 +0000 UTC m=+329.906318613" watchObservedRunningTime="2026-07-09 19:04:47.323566089 +0000 UTC m=+329.906998266"
Jul 09 19:04:47.349015 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.348957    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x" podStartSLOduration=1.466592002 podStartE2EDuration="6.348937013s" totalImagesPullingTime="4.882345011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:47.347135809 +0000 UTC m=+329.930567991" watchObservedRunningTime="2026-07-09 19:04:47.348937013 +0000 UTC m=+329.932369194"
Jul 09 19:04:47.372371 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:47.372330    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf" podStartSLOduration=2.696242072 podStartE2EDuration="8.37231674s" totalImagesPullingTime="5.676074668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:47.370906434 +0000 UTC m=+329.954338614" watchObservedRunningTime="2026-07-09 19:04:47.37231674 +0000 UTC m=+329.955748919"
Jul 09 19:04:48.298557 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:48.298525    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:52.248816 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.248783    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"]
Jul 09 19:04:52.258105 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.258080    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.263555 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.263534    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-dxl5l\""
Jul 09 19:04:52.265003 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.264984    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 09 19:04:52.277379 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.277354    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"]
Jul 09 19:04:52.296459 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.296419    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"]
Jul 09 19:04:52.299857 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.299839    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:52.302910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.302891    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 09 19:04:52.302992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.302925    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-jb4mc\""
Jul 09 19:04:52.305053 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.304981    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8273d8a6-ed4f-4d63-be7d-1b49690a8741-cert\") pod \"kserve-controller-manager-d6f4b65b8-lgpkh\" (UID: \"8273d8a6-ed4f-4d63-be7d-1b49690a8741\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.305246 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.305050    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzmkp\" (UniqueName: \"kubernetes.io/projected/8273d8a6-ed4f-4d63-be7d-1b49690a8741-kube-api-access-pzmkp\") pod \"kserve-controller-manager-d6f4b65b8-lgpkh\" (UID: \"8273d8a6-ed4f-4d63-be7d-1b49690a8741\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.310331 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.310306    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"]
Jul 09 19:04:52.405474 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.405436    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqgg7\" (UniqueName: \"kubernetes.io/projected/acddc5e0-b901-418b-88a2-56dfc9705a3a-kube-api-access-mqgg7\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:52.405649 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.405483    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:52.405649 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.405511    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8273d8a6-ed4f-4d63-be7d-1b49690a8741-cert\") pod \"kserve-controller-manager-d6f4b65b8-lgpkh\" (UID: \"8273d8a6-ed4f-4d63-be7d-1b49690a8741\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.405649 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.405567    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzmkp\" (UniqueName: \"kubernetes.io/projected/8273d8a6-ed4f-4d63-be7d-1b49690a8741-kube-api-access-pzmkp\") pod \"kserve-controller-manager-d6f4b65b8-lgpkh\" (UID: \"8273d8a6-ed4f-4d63-be7d-1b49690a8741\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.407957 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.407940    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8273d8a6-ed4f-4d63-be7d-1b49690a8741-cert\") pod \"kserve-controller-manager-d6f4b65b8-lgpkh\" (UID: \"8273d8a6-ed4f-4d63-be7d-1b49690a8741\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.415644 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.415621    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzmkp\" (UniqueName: \"kubernetes.io/projected/8273d8a6-ed4f-4d63-be7d-1b49690a8741-kube-api-access-pzmkp\") pod \"kserve-controller-manager-d6f4b65b8-lgpkh\" (UID: \"8273d8a6-ed4f-4d63-be7d-1b49690a8741\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.509834 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.506191    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:52.509834 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.506307    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqgg7\" (UniqueName: \"kubernetes.io/projected/acddc5e0-b901-418b-88a2-56dfc9705a3a-kube-api-access-mqgg7\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:52.509834 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:52.506612    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 09 19:04:52.509834 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:52.506709    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert podName:acddc5e0-b901-418b-88a2-56dfc9705a3a nodeName:}" failed. No retries permitted until 2026-07-09 19:04:53.006670247 +0000 UTC m=+335.590102409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert") pod "llmisvc-controller-manager-777c6bd7d5-4jfjt" (UID: "acddc5e0-b901-418b-88a2-56dfc9705a3a") : secret "llmisvc-webhook-server-cert" not found
Jul 09 19:04:52.517643 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.517610    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqgg7\" (UniqueName: \"kubernetes.io/projected/acddc5e0-b901-418b-88a2-56dfc9705a3a-kube-api-access-mqgg7\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:52.568303 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.568269    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:52.708123 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:52.708095    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"]
Jul 09 19:04:52.709725 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:52.709704    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8273d8a6_ed4f_4d63_be7d_1b49690a8741.slice/crio-c6214f867fb034177fc40e29143aac28865feb5291b73a36bbc7394057d3b001 WatchSource:0}: Error finding container c6214f867fb034177fc40e29143aac28865feb5291b73a36bbc7394057d3b001: Status 404 returned error can't find the container with id c6214f867fb034177fc40e29143aac28865feb5291b73a36bbc7394057d3b001
Jul 09 19:04:53.011196 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:53.011162    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:53.011361 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:53.011281    2569 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 09 19:04:53.011361 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:53.011335    2569 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert podName:acddc5e0-b901-418b-88a2-56dfc9705a3a nodeName:}" failed. No retries permitted until 2026-07-09 19:04:54.011320173 +0000 UTC m=+336.594752331 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert") pod "llmisvc-controller-manager-777c6bd7d5-4jfjt" (UID: "acddc5e0-b901-418b-88a2-56dfc9705a3a") : secret "llmisvc-webhook-server-cert" not found
Jul 09 19:04:53.320750 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:53.320658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh" event={"ID":"8273d8a6-ed4f-4d63-be7d-1b49690a8741","Type":"ContainerStarted","Data":"c6214f867fb034177fc40e29143aac28865feb5291b73a36bbc7394057d3b001"}
Jul 09 19:04:54.019256 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.019224    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:54.022111 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.022085    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/acddc5e0-b901-418b-88a2-56dfc9705a3a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-4jfjt\" (UID: \"acddc5e0-b901-418b-88a2-56dfc9705a3a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:54.112136 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.112103    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:54.302744 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.302716    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"]
Jul 09 19:04:54.303335 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:04:54.303312    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podacddc5e0_b901_418b_88a2_56dfc9705a3a.slice/crio-5a80d26106df7f27ff30286d059442693252604861fc172030d27865c167796c WatchSource:0}: Error finding container 5a80d26106df7f27ff30286d059442693252604861fc172030d27865c167796c: Status 404 returned error can't find the container with id 5a80d26106df7f27ff30286d059442693252604861fc172030d27865c167796c
Jul 09 19:04:54.305431 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.305410    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"]
Jul 09 19:04:54.305787 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.305659    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x" podUID="43bb79f9-d75d-4b7b-8451-acd79b6eea2e" containerName="discovery" containerID="cri-o://c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b" gracePeriod=30
Jul 09 19:04:54.329078 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.329044    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt" event={"ID":"acddc5e0-b901-418b-88a2-56dfc9705a3a","Type":"ContainerStarted","Data":"5a80d26106df7f27ff30286d059442693252604861fc172030d27865c167796c"}
Jul 09 19:04:54.560410 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.560353    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:54.623601 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.623559    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k6wxm\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-kube-api-access-k6wxm\") pod \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") "
Jul 09 19:04:54.623795 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.623626    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-local-certs\") pod \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") "
Jul 09 19:04:54.623795 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.623698    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-cacerts\") pod \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") "
Jul 09 19:04:54.623795 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.623734    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-dns-cert\") pod \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") "
Jul 09 19:04:54.623795 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.623789    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-ca-configmap\") pod \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") "
Jul 09 19:04:54.623985 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.623827    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-kubeconfig\") pod \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") "
Jul 09 19:04:54.623985 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.623857    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-token\") pod \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\" (UID: \"43bb79f9-d75d-4b7b-8451-acd79b6eea2e\") "
Jul 09 19:04:54.624416 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.624280    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "43bb79f9-d75d-4b7b-8451-acd79b6eea2e" (UID: "43bb79f9-d75d-4b7b-8451-acd79b6eea2e"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:04:54.626571 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.626520    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-local-certs" (OuterVolumeSpecName: "local-certs") pod "43bb79f9-d75d-4b7b-8451-acd79b6eea2e" (UID: "43bb79f9-d75d-4b7b-8451-acd79b6eea2e"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:04:54.626670 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.626595    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-kube-api-access-k6wxm" (OuterVolumeSpecName: "kube-api-access-k6wxm") pod "43bb79f9-d75d-4b7b-8451-acd79b6eea2e" (UID: "43bb79f9-d75d-4b7b-8451-acd79b6eea2e"). InnerVolumeSpecName "kube-api-access-k6wxm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:04:54.626842 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.626817    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-cacerts" (OuterVolumeSpecName: "cacerts") pod "43bb79f9-d75d-4b7b-8451-acd79b6eea2e" (UID: "43bb79f9-d75d-4b7b-8451-acd79b6eea2e"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:04:54.627091 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.627069    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "43bb79f9-d75d-4b7b-8451-acd79b6eea2e" (UID: "43bb79f9-d75d-4b7b-8451-acd79b6eea2e"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:04:54.627221 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.627200    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "43bb79f9-d75d-4b7b-8451-acd79b6eea2e" (UID: "43bb79f9-d75d-4b7b-8451-acd79b6eea2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:04:54.627463 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.627438    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "43bb79f9-d75d-4b7b-8451-acd79b6eea2e" (UID: "43bb79f9-d75d-4b7b-8451-acd79b6eea2e"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:04:54.725002 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.724969    2569 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-token\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:54.725002 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.724999    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k6wxm\" (UniqueName: \"kubernetes.io/projected/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-kube-api-access-k6wxm\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:54.725002 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.725008    2569 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-local-certs\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:54.725229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.725019    2569 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-cacerts\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:54.725229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.725028    2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-dns-cert\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:54.725229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.725036    2569 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-csr-ca-configmap\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:54.725229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:54.725043    2569 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/43bb79f9-d75d-4b7b-8451-acd79b6eea2e-istio-kubeconfig\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:04:55.336186 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.336149    2569 generic.go:356] "Generic (PLEG): container finished" podID="43bb79f9-d75d-4b7b-8451-acd79b6eea2e" containerID="c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b" exitCode=0
Jul 09 19:04:55.336656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.336186    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x" event={"ID":"43bb79f9-d75d-4b7b-8451-acd79b6eea2e","Type":"ContainerDied","Data":"c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b"}
Jul 09 19:04:55.336656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.336213    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"
Jul 09 19:04:55.336656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.336255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x" event={"ID":"43bb79f9-d75d-4b7b-8451-acd79b6eea2e","Type":"ContainerDied","Data":"4d56f80ffa71d08432f205d06f083332840ad9d4a191f875d943ef78813ebf37"}
Jul 09 19:04:55.336656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.336283    2569 scope.go:117] "RemoveContainer" containerID="c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b"
Jul 09 19:04:55.361725 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.361673    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"]
Jul 09 19:04:55.367046 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.367013    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-rjt2x"]
Jul 09 19:04:55.553435 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.553404    2569 scope.go:117] "RemoveContainer" containerID="c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b"
Jul 09 19:04:55.553811 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:04:55.553785    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b\": container with ID starting with c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b not found: ID does not exist" containerID="c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b"
Jul 09 19:04:55.553889 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.553823    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b"} err="failed to get container status \"c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b\": rpc error: code = NotFound desc = could not find container \"c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b\": container with ID starting with c101fa059607fa537eeb7dd7d4bbb7ddbbdb3d43ec60cc2243396bc222cfab5b not found: ID does not exist"
Jul 09 19:04:55.876624 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:55.876594    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43bb79f9-d75d-4b7b-8451-acd79b6eea2e" path="/var/lib/kubelet/pods/43bb79f9-d75d-4b7b-8451-acd79b6eea2e/volumes"
Jul 09 19:04:56.345957 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:56.345767    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh" event={"ID":"8273d8a6-ed4f-4d63-be7d-1b49690a8741","Type":"ContainerStarted","Data":"1d4c27379cb3a13fc9aceebf9fbdfb87b88f4b9766c22eadaecd74cb60e7f097"}
Jul 09 19:04:56.346532 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:56.346504    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:04:56.378275 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:56.378217    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh" podStartSLOduration=1.225346546 podStartE2EDuration="4.378196505s" totalImagesPullingTime="3.152849959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:56.37449976 +0000 UTC m=+338.957931964" watchObservedRunningTime="2026-07-09 19:04:56.378196505 +0000 UTC m=+338.961628686"
Jul 09 19:04:58.301417 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:58.301394    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-lv5bf"
Jul 09 19:04:58.359483 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:58.359447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt" event={"ID":"acddc5e0-b901-418b-88a2-56dfc9705a3a","Type":"ContainerStarted","Data":"13bb341a8550bce204bd8df517cae594141bf8b931c2eb1976608e5bf101b6d6"}
Jul 09 19:04:58.359870 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:58.359844    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:04:58.378548 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:58.378502    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt" podStartSLOduration=2.458433032 podStartE2EDuration="6.378487455s" totalImagesPullingTime="3.920054423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:04:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:04:58.377920792 +0000 UTC m=+340.961352974" watchObservedRunningTime="2026-07-09 19:04:58.378487455 +0000 UTC m=+340.961919665"
Jul 09 19:04:59.306208 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:04:59.306177    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-rbj5f"
Jul 09 19:05:28.365021 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:28.364992    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-lgpkh"
Jul 09 19:05:30.372206 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:30.372174    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-4jfjt"
Jul 09 19:05:35.492207 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.492173    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r5wmp"]
Jul 09 19:05:35.492570 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.492504    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43bb79f9-d75d-4b7b-8451-acd79b6eea2e" containerName="discovery"
Jul 09 19:05:35.492570 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.492515    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="43bb79f9-d75d-4b7b-8451-acd79b6eea2e" containerName="discovery"
Jul 09 19:05:35.492642 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.492573    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="43bb79f9-d75d-4b7b-8451-acd79b6eea2e" containerName="discovery"
Jul 09 19:05:35.503194 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.503170    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp"
Jul 09 19:05:35.506454 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.506432    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 09 19:05:35.507850 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.507827    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 09 19:05:35.508026 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.507962    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kdrrs\""
Jul 09 19:05:35.511729 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.511699    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r5wmp"]
Jul 09 19:05:35.674097 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.674065    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dztlm\" (UniqueName: \"kubernetes.io/projected/7e348e7b-bf8b-450e-b4b5-51bc7d5bb688-kube-api-access-dztlm\") pod \"kuadrant-operator-catalog-r5wmp\" (UID: \"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688\") " pod="kuadrant-system/kuadrant-operator-catalog-r5wmp"
Jul 09 19:05:35.774864 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.774776    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dztlm\" (UniqueName: \"kubernetes.io/projected/7e348e7b-bf8b-450e-b4b5-51bc7d5bb688-kube-api-access-dztlm\") pod \"kuadrant-operator-catalog-r5wmp\" (UID: \"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688\") " pod="kuadrant-system/kuadrant-operator-catalog-r5wmp"
Jul 09 19:05:35.784353 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.784332    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dztlm\" (UniqueName: \"kubernetes.io/projected/7e348e7b-bf8b-450e-b4b5-51bc7d5bb688-kube-api-access-dztlm\") pod \"kuadrant-operator-catalog-r5wmp\" (UID: \"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688\") " pod="kuadrant-system/kuadrant-operator-catalog-r5wmp"
Jul 09 19:05:35.814724 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.814667    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp"
Jul 09 19:05:35.827667 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.827637    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r5wmp"]
Jul 09 19:05:35.943882 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:35.943855    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r5wmp"]
Jul 09 19:05:35.947312 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:05:35.947284    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e348e7b_bf8b_450e_b4b5_51bc7d5bb688.slice/crio-ac1d580f3f42569c70d3bebf7af4576c6547ead1e6d5c579f64f687493cbc320 WatchSource:0}: Error finding container ac1d580f3f42569c70d3bebf7af4576c6547ead1e6d5c579f64f687493cbc320: Status 404 returned error can't find the container with id ac1d580f3f42569c70d3bebf7af4576c6547ead1e6d5c579f64f687493cbc320
Jul 09 19:05:36.036111 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.036044    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hklx6"]
Jul 09 19:05:36.038876 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.038860    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:36.046538 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.046517    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hklx6"]
Jul 09 19:05:36.177998 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.177968    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wprf\" (UniqueName: \"kubernetes.io/projected/1e5e329a-afaa-41cc-b0d4-7c3b27ff4826-kube-api-access-6wprf\") pod \"kuadrant-operator-catalog-hklx6\" (UID: \"1e5e329a-afaa-41cc-b0d4-7c3b27ff4826\") " pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:36.279106 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.279071    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wprf\" (UniqueName: \"kubernetes.io/projected/1e5e329a-afaa-41cc-b0d4-7c3b27ff4826-kube-api-access-6wprf\") pod \"kuadrant-operator-catalog-hklx6\" (UID: \"1e5e329a-afaa-41cc-b0d4-7c3b27ff4826\") " pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:36.287509 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.287448    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wprf\" (UniqueName: \"kubernetes.io/projected/1e5e329a-afaa-41cc-b0d4-7c3b27ff4826-kube-api-access-6wprf\") pod \"kuadrant-operator-catalog-hklx6\" (UID: \"1e5e329a-afaa-41cc-b0d4-7c3b27ff4826\") " pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:36.349297 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.349270    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:36.476252 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.476230    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hklx6"]
Jul 09 19:05:36.478451 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:05:36.478426    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5e329a_afaa_41cc_b0d4_7c3b27ff4826.slice/crio-cd47ea037776c8711ae0acfd0cf5383f27d5c0db54eaa57f135d9707b84c1705 WatchSource:0}: Error finding container cd47ea037776c8711ae0acfd0cf5383f27d5c0db54eaa57f135d9707b84c1705: Status 404 returned error can't find the container with id cd47ea037776c8711ae0acfd0cf5383f27d5c0db54eaa57f135d9707b84c1705
Jul 09 19:05:36.535659 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.535626    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hklx6" event={"ID":"1e5e329a-afaa-41cc-b0d4-7c3b27ff4826","Type":"ContainerStarted","Data":"cd47ea037776c8711ae0acfd0cf5383f27d5c0db54eaa57f135d9707b84c1705"}
Jul 09 19:05:36.536716 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:36.536672    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp" event={"ID":"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688","Type":"ContainerStarted","Data":"ac1d580f3f42569c70d3bebf7af4576c6547ead1e6d5c579f64f687493cbc320"}
Jul 09 19:05:38.550296 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.550211    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hklx6" event={"ID":"1e5e329a-afaa-41cc-b0d4-7c3b27ff4826","Type":"ContainerStarted","Data":"d4f088e7dafccbecb1501b07940e5102e85825385f936187b40d859ac0632544"}
Jul 09 19:05:38.557345 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.557314    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp" event={"ID":"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688","Type":"ContainerStarted","Data":"7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e"}
Jul 09 19:05:38.557513 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.557404    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp" podUID="7e348e7b-bf8b-450e-b4b5-51bc7d5bb688" containerName="registry-server" containerID="cri-o://7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e" gracePeriod=2
Jul 09 19:05:38.568752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.568703    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hklx6" podStartSLOduration=0.843782179 podStartE2EDuration="2.568667055s" totalImagesPullingTime="1.724884876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:05:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:05:38.565857709 +0000 UTC m=+381.149289889" watchObservedRunningTime="2026-07-09 19:05:38.568667055 +0000 UTC m=+381.152099239"
Jul 09 19:05:38.581292 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.581251    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp" podStartSLOduration=1.325776272 podStartE2EDuration="3.581238022s" totalImagesPullingTime="2.25546175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:05:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:05:38.57998914 +0000 UTC m=+381.163421321" watchObservedRunningTime="2026-07-09 19:05:38.581238022 +0000 UTC m=+381.164670202"
Jul 09 19:05:38.796928 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.796900    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp"
Jul 09 19:05:38.904251 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.904223    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dztlm\" (UniqueName: \"kubernetes.io/projected/7e348e7b-bf8b-450e-b4b5-51bc7d5bb688-kube-api-access-dztlm\") pod \"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688\" (UID: \"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688\") "
Jul 09 19:05:38.906477 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:38.906456    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e348e7b-bf8b-450e-b4b5-51bc7d5bb688-kube-api-access-dztlm" (OuterVolumeSpecName: "kube-api-access-dztlm") pod "7e348e7b-bf8b-450e-b4b5-51bc7d5bb688" (UID: "7e348e7b-bf8b-450e-b4b5-51bc7d5bb688"). InnerVolumeSpecName "kube-api-access-dztlm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:05:39.005657 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.005618    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dztlm\" (UniqueName: \"kubernetes.io/projected/7e348e7b-bf8b-450e-b4b5-51bc7d5bb688-kube-api-access-dztlm\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:39.562586 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.562498    2569 generic.go:356] "Generic (PLEG): container finished" podID="7e348e7b-bf8b-450e-b4b5-51bc7d5bb688" containerID="7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e" exitCode=0
Jul 09 19:05:39.562586 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.562536    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp" event={"ID":"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688","Type":"ContainerDied","Data":"7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e"}
Jul 09 19:05:39.563083 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.562599    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp" event={"ID":"7e348e7b-bf8b-450e-b4b5-51bc7d5bb688","Type":"ContainerDied","Data":"ac1d580f3f42569c70d3bebf7af4576c6547ead1e6d5c579f64f687493cbc320"}
Jul 09 19:05:39.563083 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.562626    2569 scope.go:117] "RemoveContainer" containerID="7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e"
Jul 09 19:05:39.563083 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.562549    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-r5wmp"
Jul 09 19:05:39.571927 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.571912    2569 scope.go:117] "RemoveContainer" containerID="7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e"
Jul 09 19:05:39.572151 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:05:39.572131    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e\": container with ID starting with 7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e not found: ID does not exist" containerID="7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e"
Jul 09 19:05:39.572210 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.572163    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e"} err="failed to get container status \"7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e\": rpc error: code = NotFound desc = could not find container \"7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e\": container with ID starting with 7f1ba1b452a20aa2223bf138adf64a13ff9a772836e47bd15b81aa7161bb902e not found: ID does not exist"
Jul 09 19:05:39.583906 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.583884    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r5wmp"]
Jul 09 19:05:39.588223 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.588203    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-r5wmp"]
Jul 09 19:05:39.877007 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:39.876977    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e348e7b-bf8b-450e-b4b5-51bc7d5bb688" path="/var/lib/kubelet/pods/7e348e7b-bf8b-450e-b4b5-51bc7d5bb688/volumes"
Jul 09 19:05:46.350192 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:46.350162    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:46.350611 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:46.350409    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:46.373911 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:46.373887    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:46.616968 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:46.616943    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hklx6"
Jul 09 19:05:50.822241 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.822202    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"]
Jul 09 19:05:50.822640 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.822525    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e348e7b-bf8b-450e-b4b5-51bc7d5bb688" containerName="registry-server"
Jul 09 19:05:50.822640 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.822538    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e348e7b-bf8b-450e-b4b5-51bc7d5bb688" containerName="registry-server"
Jul 09 19:05:50.822640 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.822599    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e348e7b-bf8b-450e-b4b5-51bc7d5bb688" containerName="registry-server"
Jul 09 19:05:50.828089 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.828064    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:50.832289 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.832236    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-njbf7\""
Jul 09 19:05:50.833038 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.833016    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"]
Jul 09 19:05:50.902267 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.902231    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:50.902267 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.902264    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5vkr\" (UniqueName: \"kubernetes.io/projected/7b10431a-04b9-4b7c-909d-7d67bf556c50-kube-api-access-m5vkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:50.902619 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:50.902594    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.003247 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.003212    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.003389 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.003258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.003389 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.003275    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5vkr\" (UniqueName: \"kubernetes.io/projected/7b10431a-04b9-4b7c-909d-7d67bf556c50-kube-api-access-m5vkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.003604 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.003585    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.003640 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.003600    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.013002 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.012969    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5vkr\" (UniqueName: \"kubernetes.io/projected/7b10431a-04b9-4b7c-909d-7d67bf556c50-kube-api-access-m5vkr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.139003 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.138975    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:51.277954 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.277930    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"]
Jul 09 19:05:51.280300 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:05:51.280271    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b10431a_04b9_4b7c_909d_7d67bf556c50.slice/crio-d1b33de13a5ae81942c69c238f44e34f6c0a4b1818307dcba70fc0420440c803 WatchSource:0}: Error finding container d1b33de13a5ae81942c69c238f44e34f6c0a4b1818307dcba70fc0420440c803: Status 404 returned error can't find the container with id d1b33de13a5ae81942c69c238f44e34f6c0a4b1818307dcba70fc0420440c803
Jul 09 19:05:51.420891 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.420827    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"]
Jul 09 19:05:51.423909 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.423889    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.431888 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.431863    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"]
Jul 09 19:05:51.507997 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.507969    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.508103 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.508003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.508103 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.508031    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbcfp\" (UniqueName: \"kubernetes.io/projected/9d237620-7db5-4607-b1c1-a9d054e3c05b-kube-api-access-sbcfp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.608947 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.608919    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.609071 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.608959    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.609071 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.608988    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbcfp\" (UniqueName: \"kubernetes.io/projected/9d237620-7db5-4607-b1c1-a9d054e3c05b-kube-api-access-sbcfp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.609288 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.609271    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.609327 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.609309    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.617255 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.617233    2569 generic.go:356] "Generic (PLEG): container finished" podID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerID="54b3dfb6f4faad62702b20858917c08c2f0949d3ecee0d4f5b9005b202c6e6ab" exitCode=0
Jul 09 19:05:51.617355 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.617263    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f" event={"ID":"7b10431a-04b9-4b7c-909d-7d67bf556c50","Type":"ContainerDied","Data":"54b3dfb6f4faad62702b20858917c08c2f0949d3ecee0d4f5b9005b202c6e6ab"}
Jul 09 19:05:51.617355 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.617298    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f" event={"ID":"7b10431a-04b9-4b7c-909d-7d67bf556c50","Type":"ContainerStarted","Data":"d1b33de13a5ae81942c69c238f44e34f6c0a4b1818307dcba70fc0420440c803"}
Jul 09 19:05:51.619209 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.619190    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbcfp\" (UniqueName: \"kubernetes.io/projected/9d237620-7db5-4607-b1c1-a9d054e3c05b-kube-api-access-sbcfp\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.754333 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.754262    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:51.824407 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.824349    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"]
Jul 09 19:05:51.827892 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.827873    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:51.836243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.836220    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"]
Jul 09 19:05:51.892769 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.892747    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"]
Jul 09 19:05:51.894421 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:05:51.894398    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d237620_7db5_4607_b1c1_a9d054e3c05b.slice/crio-27b0c687391fafc49eaf77a89b6e3ac41c2a7bd46768dd4906161f7c0740ec25 WatchSource:0}: Error finding container 27b0c687391fafc49eaf77a89b6e3ac41c2a7bd46768dd4906161f7c0740ec25: Status 404 returned error can't find the container with id 27b0c687391fafc49eaf77a89b6e3ac41c2a7bd46768dd4906161f7c0740ec25
Jul 09 19:05:51.912203 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.912180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:51.912296 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.912255    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2tm8\" (UniqueName: \"kubernetes.io/projected/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-kube-api-access-k2tm8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:51.912338 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:51.912310    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.013361 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.013300    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2tm8\" (UniqueName: \"kubernetes.io/projected/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-kube-api-access-k2tm8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.013361 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.013347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.013519 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.013399    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.013747 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.013730    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.013793 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.013774    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.022239 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.022219    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2tm8\" (UniqueName: \"kubernetes.io/projected/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-kube-api-access-k2tm8\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.139380 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.139352    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:52.229698 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.229651    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"]
Jul 09 19:05:52.233820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.233753    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.243914 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.242028    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"]
Jul 09 19:05:52.287398 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.287331    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"]
Jul 09 19:05:52.305879 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:05:52.305851    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf2c15b7_0dbe_4fee_adea_df55bfbb74b7.slice/crio-fc982b702ea4e0f1b6dd2900ab0affb48f246b08b32e11d21e9a03b1b8bcaad6 WatchSource:0}: Error finding container fc982b702ea4e0f1b6dd2900ab0affb48f246b08b32e11d21e9a03b1b8bcaad6: Status 404 returned error can't find the container with id fc982b702ea4e0f1b6dd2900ab0affb48f246b08b32e11d21e9a03b1b8bcaad6
Jul 09 19:05:52.315715 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.315675    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xbdw\" (UniqueName: \"kubernetes.io/projected/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-kube-api-access-5xbdw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.315765 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.315737    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.315765 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.315761    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.416465 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.416437    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.416574 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.416481    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.416574 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.416542    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xbdw\" (UniqueName: \"kubernetes.io/projected/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-kube-api-access-5xbdw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.416832 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.416813    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.417131 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.417113    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.424973 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.424944    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xbdw\" (UniqueName: \"kubernetes.io/projected/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-kube-api-access-5xbdw\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.547767 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.547661    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:52.623408 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.623378    2569 generic.go:356] "Generic (PLEG): container finished" podID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerID="711d4404ff2f23ceb931c77fc6896a0fe5454d09d79e6bfadb0408b017574c34" exitCode=0
Jul 09 19:05:52.623566 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.623462    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts" event={"ID":"df2c15b7-0dbe-4fee-adea-df55bfbb74b7","Type":"ContainerDied","Data":"711d4404ff2f23ceb931c77fc6896a0fe5454d09d79e6bfadb0408b017574c34"}
Jul 09 19:05:52.623566 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.623522    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts" event={"ID":"df2c15b7-0dbe-4fee-adea-df55bfbb74b7","Type":"ContainerStarted","Data":"fc982b702ea4e0f1b6dd2900ab0affb48f246b08b32e11d21e9a03b1b8bcaad6"}
Jul 09 19:05:52.625241 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.625200    2569 generic.go:356] "Generic (PLEG): container finished" podID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerID="1ffc28d071f390227565bb1deaff96afa7610b00a3bdedddc851f58aba062446" exitCode=0
Jul 09 19:05:52.625376 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.625267    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8" event={"ID":"9d237620-7db5-4607-b1c1-a9d054e3c05b","Type":"ContainerDied","Data":"1ffc28d071f390227565bb1deaff96afa7610b00a3bdedddc851f58aba062446"}
Jul 09 19:05:52.625376 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.625316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8" event={"ID":"9d237620-7db5-4607-b1c1-a9d054e3c05b","Type":"ContainerStarted","Data":"27b0c687391fafc49eaf77a89b6e3ac41c2a7bd46768dd4906161f7c0740ec25"}
Jul 09 19:05:52.627438 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.627338    2569 generic.go:356] "Generic (PLEG): container finished" podID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerID="0a66b78e5ce8e750f656e23fe0926b7ca10bd1a3d6f87b56bc389485c6f53401" exitCode=0
Jul 09 19:05:52.627548 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.627474    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f" event={"ID":"7b10431a-04b9-4b7c-909d-7d67bf556c50","Type":"ContainerDied","Data":"0a66b78e5ce8e750f656e23fe0926b7ca10bd1a3d6f87b56bc389485c6f53401"}
Jul 09 19:05:52.692045 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:52.692020    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"]
Jul 09 19:05:52.693182 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:05:52.693160    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d6ff828_0473_4a21_a38f_0164ca5b4d9a.slice/crio-f8edc7b81ee6a0a17aaded04868356a5140cf2a8d78fb2db5d328fa445fc4ec3 WatchSource:0}: Error finding container f8edc7b81ee6a0a17aaded04868356a5140cf2a8d78fb2db5d328fa445fc4ec3: Status 404 returned error can't find the container with id f8edc7b81ee6a0a17aaded04868356a5140cf2a8d78fb2db5d328fa445fc4ec3
Jul 09 19:05:53.633625 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:53.633591    2569 generic.go:356] "Generic (PLEG): container finished" podID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerID="3c81bab4ab1b4a4c224126caefa64dd784023f16e6ade094ec7e22375637e587" exitCode=0
Jul 09 19:05:53.634111 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:53.633692    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts" event={"ID":"df2c15b7-0dbe-4fee-adea-df55bfbb74b7","Type":"ContainerDied","Data":"3c81bab4ab1b4a4c224126caefa64dd784023f16e6ade094ec7e22375637e587"}
Jul 09 19:05:53.635051 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:53.635032    2569 generic.go:356] "Generic (PLEG): container finished" podID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerID="413983b35f292a8564be838d8e7b9cd66f55df7177b1f467ca5f253a2eb22e19" exitCode=0
Jul 09 19:05:53.635168 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:53.635107    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m" event={"ID":"1d6ff828-0473-4a21-a38f-0164ca5b4d9a","Type":"ContainerDied","Data":"413983b35f292a8564be838d8e7b9cd66f55df7177b1f467ca5f253a2eb22e19"}
Jul 09 19:05:53.635168 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:53.635143    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m" event={"ID":"1d6ff828-0473-4a21-a38f-0164ca5b4d9a","Type":"ContainerStarted","Data":"f8edc7b81ee6a0a17aaded04868356a5140cf2a8d78fb2db5d328fa445fc4ec3"}
Jul 09 19:05:53.637271 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:53.637247    2569 generic.go:356] "Generic (PLEG): container finished" podID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerID="52aaa030392c9c8e6eb667e993ba1f5da15a6444636751d618c2395ef1415c83" exitCode=0
Jul 09 19:05:53.637416 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:53.637393    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f" event={"ID":"7b10431a-04b9-4b7c-909d-7d67bf556c50","Type":"ContainerDied","Data":"52aaa030392c9c8e6eb667e993ba1f5da15a6444636751d618c2395ef1415c83"}
Jul 09 19:05:54.644564 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.644532    2569 generic.go:356] "Generic (PLEG): container finished" podID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerID="56c8b13c4e2e3d26b8d7298d0ed76f2fe2dc22e99bee13d0c2bcdf103dcc16b4" exitCode=0
Jul 09 19:05:54.645007 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.644616    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts" event={"ID":"df2c15b7-0dbe-4fee-adea-df55bfbb74b7","Type":"ContainerDied","Data":"56c8b13c4e2e3d26b8d7298d0ed76f2fe2dc22e99bee13d0c2bcdf103dcc16b4"}
Jul 09 19:05:54.646370 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.646350    2569 generic.go:356] "Generic (PLEG): container finished" podID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerID="2b6e87ab8c74e7309ef392dc862182906e7e29534698f6a647c893a77bc4ee05" exitCode=0
Jul 09 19:05:54.646490 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.646422    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8" event={"ID":"9d237620-7db5-4607-b1c1-a9d054e3c05b","Type":"ContainerDied","Data":"2b6e87ab8c74e7309ef392dc862182906e7e29534698f6a647c893a77bc4ee05"}
Jul 09 19:05:54.648500 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.648476    2569 generic.go:356] "Generic (PLEG): container finished" podID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerID="2f100c7b34d17f330112b9e8163f9f1d69baf002691ce78c9987b827fcee652e" exitCode=0
Jul 09 19:05:54.648594 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.648559    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m" event={"ID":"1d6ff828-0473-4a21-a38f-0164ca5b4d9a","Type":"ContainerDied","Data":"2f100c7b34d17f330112b9e8163f9f1d69baf002691ce78c9987b827fcee652e"}
Jul 09 19:05:54.785286 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.785263    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:54.838958 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.838935    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-bundle\") pod \"7b10431a-04b9-4b7c-909d-7d67bf556c50\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") "
Jul 09 19:05:54.839052 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.838978    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5vkr\" (UniqueName: \"kubernetes.io/projected/7b10431a-04b9-4b7c-909d-7d67bf556c50-kube-api-access-m5vkr\") pod \"7b10431a-04b9-4b7c-909d-7d67bf556c50\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") "
Jul 09 19:05:54.839097 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.839056    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-util\") pod \"7b10431a-04b9-4b7c-909d-7d67bf556c50\" (UID: \"7b10431a-04b9-4b7c-909d-7d67bf556c50\") "
Jul 09 19:05:54.839542 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.839515    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-bundle" (OuterVolumeSpecName: "bundle") pod "7b10431a-04b9-4b7c-909d-7d67bf556c50" (UID: "7b10431a-04b9-4b7c-909d-7d67bf556c50"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:54.841123 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.841098    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b10431a-04b9-4b7c-909d-7d67bf556c50-kube-api-access-m5vkr" (OuterVolumeSpecName: "kube-api-access-m5vkr") pod "7b10431a-04b9-4b7c-909d-7d67bf556c50" (UID: "7b10431a-04b9-4b7c-909d-7d67bf556c50"). InnerVolumeSpecName "kube-api-access-m5vkr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:05:54.844407 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.844387    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-util" (OuterVolumeSpecName: "util") pod "7b10431a-04b9-4b7c-909d-7d67bf556c50" (UID: "7b10431a-04b9-4b7c-909d-7d67bf556c50"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:54.939735 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.939708    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-util\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:54.939735 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.939731    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7b10431a-04b9-4b7c-909d-7d67bf556c50-bundle\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:54.939735 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:54.939740    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5vkr\" (UniqueName: \"kubernetes.io/projected/7b10431a-04b9-4b7c-909d-7d67bf556c50-kube-api-access-m5vkr\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:55.656320 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.656269    2569 generic.go:356] "Generic (PLEG): container finished" podID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerID="c418d19906cccce2a97f4e6221747b95c84a8142237d7cb11768b4911404beb0" exitCode=0
Jul 09 19:05:55.656783 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.656366    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8" event={"ID":"9d237620-7db5-4607-b1c1-a9d054e3c05b","Type":"ContainerDied","Data":"c418d19906cccce2a97f4e6221747b95c84a8142237d7cb11768b4911404beb0"}
Jul 09 19:05:55.658403 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.658373    2569 generic.go:356] "Generic (PLEG): container finished" podID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerID="ce4bd92eb29094eb8d02482f54c80fef056d15774b64f5a208ab03f0c7e52e13" exitCode=0
Jul 09 19:05:55.658530 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.658446    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m" event={"ID":"1d6ff828-0473-4a21-a38f-0164ca5b4d9a","Type":"ContainerDied","Data":"ce4bd92eb29094eb8d02482f54c80fef056d15774b64f5a208ab03f0c7e52e13"}
Jul 09 19:05:55.660242 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.660218    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f" event={"ID":"7b10431a-04b9-4b7c-909d-7d67bf556c50","Type":"ContainerDied","Data":"d1b33de13a5ae81942c69c238f44e34f6c0a4b1818307dcba70fc0420440c803"}
Jul 09 19:05:55.660328 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.660257    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1b33de13a5ae81942c69c238f44e34f6c0a4b1818307dcba70fc0420440c803"
Jul 09 19:05:55.660415 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.660400    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f"
Jul 09 19:05:55.794189 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.794165    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:55.847468 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.847434    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2tm8\" (UniqueName: \"kubernetes.io/projected/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-kube-api-access-k2tm8\") pod \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") "
Jul 09 19:05:55.847634 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.847479    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-util\") pod \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") "
Jul 09 19:05:55.847634 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.847586    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-bundle\") pod \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\" (UID: \"df2c15b7-0dbe-4fee-adea-df55bfbb74b7\") "
Jul 09 19:05:55.848136 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.848097    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-bundle" (OuterVolumeSpecName: "bundle") pod "df2c15b7-0dbe-4fee-adea-df55bfbb74b7" (UID: "df2c15b7-0dbe-4fee-adea-df55bfbb74b7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:55.849832 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.849811    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-kube-api-access-k2tm8" (OuterVolumeSpecName: "kube-api-access-k2tm8") pod "df2c15b7-0dbe-4fee-adea-df55bfbb74b7" (UID: "df2c15b7-0dbe-4fee-adea-df55bfbb74b7"). InnerVolumeSpecName "kube-api-access-k2tm8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:05:55.853183 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.853165    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-util" (OuterVolumeSpecName: "util") pod "df2c15b7-0dbe-4fee-adea-df55bfbb74b7" (UID: "df2c15b7-0dbe-4fee-adea-df55bfbb74b7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:55.948927 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.948850    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-bundle\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:55.948927 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.948877    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2tm8\" (UniqueName: \"kubernetes.io/projected/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-kube-api-access-k2tm8\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:55.948927 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:55.948889    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/df2c15b7-0dbe-4fee-adea-df55bfbb74b7-util\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:56.666316 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.666285    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts"
Jul 09 19:05:56.666316 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.666304    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts" event={"ID":"df2c15b7-0dbe-4fee-adea-df55bfbb74b7","Type":"ContainerDied","Data":"fc982b702ea4e0f1b6dd2900ab0affb48f246b08b32e11d21e9a03b1b8bcaad6"}
Jul 09 19:05:56.666756 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.666344    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc982b702ea4e0f1b6dd2900ab0affb48f246b08b32e11d21e9a03b1b8bcaad6"
Jul 09 19:05:56.826956 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.826933    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:56.829001 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.828984    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:56.957476 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.957387    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xbdw\" (UniqueName: \"kubernetes.io/projected/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-kube-api-access-5xbdw\") pod \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") "
Jul 09 19:05:56.957476 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.957429    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbcfp\" (UniqueName: \"kubernetes.io/projected/9d237620-7db5-4607-b1c1-a9d054e3c05b-kube-api-access-sbcfp\") pod \"9d237620-7db5-4607-b1c1-a9d054e3c05b\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") "
Jul 09 19:05:56.957476 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.957462    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-bundle\") pod \"9d237620-7db5-4607-b1c1-a9d054e3c05b\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") "
Jul 09 19:05:56.957764 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.957533    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-bundle\") pod \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") "
Jul 09 19:05:56.957764 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.957574    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-util\") pod \"9d237620-7db5-4607-b1c1-a9d054e3c05b\" (UID: \"9d237620-7db5-4607-b1c1-a9d054e3c05b\") "
Jul 09 19:05:56.957764 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.957609    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-util\") pod \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\" (UID: \"1d6ff828-0473-4a21-a38f-0164ca5b4d9a\") "
Jul 09 19:05:56.958109 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.958060    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-bundle" (OuterVolumeSpecName: "bundle") pod "1d6ff828-0473-4a21-a38f-0164ca5b4d9a" (UID: "1d6ff828-0473-4a21-a38f-0164ca5b4d9a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:56.958396 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.958362    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-bundle" (OuterVolumeSpecName: "bundle") pod "9d237620-7db5-4607-b1c1-a9d054e3c05b" (UID: "9d237620-7db5-4607-b1c1-a9d054e3c05b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:56.959784 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.959746    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d237620-7db5-4607-b1c1-a9d054e3c05b-kube-api-access-sbcfp" (OuterVolumeSpecName: "kube-api-access-sbcfp") pod "9d237620-7db5-4607-b1c1-a9d054e3c05b" (UID: "9d237620-7db5-4607-b1c1-a9d054e3c05b"). InnerVolumeSpecName "kube-api-access-sbcfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:05:56.960030 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.960007    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-kube-api-access-5xbdw" (OuterVolumeSpecName: "kube-api-access-5xbdw") pod "1d6ff828-0473-4a21-a38f-0164ca5b4d9a" (UID: "1d6ff828-0473-4a21-a38f-0164ca5b4d9a"). InnerVolumeSpecName "kube-api-access-5xbdw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:05:56.963722 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.963697    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-util" (OuterVolumeSpecName: "util") pod "1d6ff828-0473-4a21-a38f-0164ca5b4d9a" (UID: "1d6ff828-0473-4a21-a38f-0164ca5b4d9a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:56.964164 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:56.964143    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-util" (OuterVolumeSpecName: "util") pod "9d237620-7db5-4607-b1c1-a9d054e3c05b" (UID: "9d237620-7db5-4607-b1c1-a9d054e3c05b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:05:57.058353 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.058326    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xbdw\" (UniqueName: \"kubernetes.io/projected/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-kube-api-access-5xbdw\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:57.058353 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.058348    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbcfp\" (UniqueName: \"kubernetes.io/projected/9d237620-7db5-4607-b1c1-a9d054e3c05b-kube-api-access-sbcfp\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:57.058484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.058359    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-bundle\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:57.058484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.058369    2569 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-bundle\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:57.058484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.058377    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9d237620-7db5-4607-b1c1-a9d054e3c05b-util\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:57.058484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.058385    2569 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d6ff828-0473-4a21-a38f-0164ca5b4d9a-util\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:05:57.673347 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.673305    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8" event={"ID":"9d237620-7db5-4607-b1c1-a9d054e3c05b","Type":"ContainerDied","Data":"27b0c687391fafc49eaf77a89b6e3ac41c2a7bd46768dd4906161f7c0740ec25"}
Jul 09 19:05:57.673779 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.673365    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="27b0c687391fafc49eaf77a89b6e3ac41c2a7bd46768dd4906161f7c0740ec25"
Jul 09 19:05:57.673779 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.673328    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8"
Jul 09 19:05:57.675312 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.675293    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m"
Jul 09 19:05:57.675417 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.675316    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m" event={"ID":"1d6ff828-0473-4a21-a38f-0164ca5b4d9a","Type":"ContainerDied","Data":"f8edc7b81ee6a0a17aaded04868356a5140cf2a8d78fb2db5d328fa445fc4ec3"}
Jul 09 19:05:57.675417 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:05:57.675354    2569 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8edc7b81ee6a0a17aaded04868356a5140cf2a8d78fb2db5d328fa445fc4ec3"
Jul 09 19:06:06.872816 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.872779    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"]
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873112    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873123    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873137    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873143    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873156    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerName="pull"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873161    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerName="pull"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873170    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873174    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873181    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873185    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873191    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873196    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerName="extract"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873201    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerName="pull"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873210    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerName="pull"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873217    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerName="pull"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873222    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerName="pull"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873228    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873232    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873236    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873241    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873246    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873251    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerName="util"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873256    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerName="pull"
Jul 09 19:06:06.873290 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873265    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerName="pull"
Jul 09 19:06:06.873996 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873336    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="df2c15b7-0dbe-4fee-adea-df55bfbb74b7" containerName="extract"
Jul 09 19:06:06.873996 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873344    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d6ff828-0473-4a21-a38f-0164ca5b4d9a" containerName="extract"
Jul 09 19:06:06.873996 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873352    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b10431a-04b9-4b7c-909d-7d67bf556c50" containerName="extract"
Jul 09 19:06:06.873996 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.873358    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d237620-7db5-4607-b1c1-a9d054e3c05b" containerName="extract"
Jul 09 19:06:06.880138 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.878251    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"
Jul 09 19:06:06.882960 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.882934    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-84dhw\""
Jul 09 19:06:06.883096 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.883080    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 09 19:06:06.887831 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.887810    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"]
Jul 09 19:06:06.943028 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:06.943001    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9wsr\" (UniqueName: \"kubernetes.io/projected/138c1a2e-aca2-4c06-8625-8b4bc25f28ad-kube-api-access-c9wsr\") pod \"dns-operator-controller-manager-864dd78c4c-whw9v\" (UID: \"138c1a2e-aca2-4c06-8625-8b4bc25f28ad\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"
Jul 09 19:06:07.043570 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:07.043532    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9wsr\" (UniqueName: \"kubernetes.io/projected/138c1a2e-aca2-4c06-8625-8b4bc25f28ad-kube-api-access-c9wsr\") pod \"dns-operator-controller-manager-864dd78c4c-whw9v\" (UID: \"138c1a2e-aca2-4c06-8625-8b4bc25f28ad\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"
Jul 09 19:06:07.060093 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:07.060065    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9wsr\" (UniqueName: \"kubernetes.io/projected/138c1a2e-aca2-4c06-8625-8b4bc25f28ad-kube-api-access-c9wsr\") pod \"dns-operator-controller-manager-864dd78c4c-whw9v\" (UID: \"138c1a2e-aca2-4c06-8625-8b4bc25f28ad\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"
Jul 09 19:06:07.189755 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:07.189656    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"
Jul 09 19:06:07.326167 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:07.326142    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"]
Jul 09 19:06:07.327967 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:06:07.327940    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod138c1a2e_aca2_4c06_8625_8b4bc25f28ad.slice/crio-ae96462205e2e2be422679ea3293dc0b87d691768a1d22ef75c6c87659d4c947 WatchSource:0}: Error finding container ae96462205e2e2be422679ea3293dc0b87d691768a1d22ef75c6c87659d4c947: Status 404 returned error can't find the container with id ae96462205e2e2be422679ea3293dc0b87d691768a1d22ef75c6c87659d4c947
Jul 09 19:06:07.726514 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:07.726482    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v" event={"ID":"138c1a2e-aca2-4c06-8625-8b4bc25f28ad","Type":"ContainerStarted","Data":"ae96462205e2e2be422679ea3293dc0b87d691768a1d22ef75c6c87659d4c947"}
Jul 09 19:06:09.739012 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:09.738977    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v" event={"ID":"138c1a2e-aca2-4c06-8625-8b4bc25f28ad","Type":"ContainerStarted","Data":"5398bd8f6e98f7a38e829e11704c7fb0f73ed34c6e4cee05bbe78564b872392b"}
Jul 09 19:06:09.739370 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:09.739099    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"
Jul 09 19:06:09.758033 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:09.757986    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v" podStartSLOduration=1.766480276 podStartE2EDuration="3.757972234s" totalImagesPullingTime="1.991491958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:06:09.75575489 +0000 UTC m=+412.339187072" watchObservedRunningTime="2026-07-09 19:06:09.757972234 +0000 UTC m=+412.341404415"
Jul 09 19:06:12.287582 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.287545    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"]
Jul 09 19:06:12.295502 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.295481    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.300179 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.300159    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mh8f8\""
Jul 09 19:06:12.326805 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.326782    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"]
Jul 09 19:06:12.392339 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.392304    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36c2b9e-12bf-470e-8aa0-b14db68626cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.392525 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.392358    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w72l\" (UniqueName: \"kubernetes.io/projected/c36c2b9e-12bf-470e-8aa0-b14db68626cb-kube-api-access-8w72l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.492844 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.492811    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36c2b9e-12bf-470e-8aa0-b14db68626cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.493029 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.492881    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w72l\" (UniqueName: \"kubernetes.io/projected/c36c2b9e-12bf-470e-8aa0-b14db68626cb-kube-api-access-8w72l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.493266 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.493240    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36c2b9e-12bf-470e-8aa0-b14db68626cb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.506802 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.506777    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w72l\" (UniqueName: \"kubernetes.io/projected/c36c2b9e-12bf-470e-8aa0-b14db68626cb-kube-api-access-8w72l\") pod \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.605980 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.605899    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:12.770450 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:12.770422    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"]
Jul 09 19:06:12.771210 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:06:12.771188    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc36c2b9e_12bf_470e_8aa0_b14db68626cb.slice/crio-3cd64fc7f5d91ecf71ce1c0aef9daa348fd7a2d57555e5d77acd9d16633ca645 WatchSource:0}: Error finding container 3cd64fc7f5d91ecf71ce1c0aef9daa348fd7a2d57555e5d77acd9d16633ca645: Status 404 returned error can't find the container with id 3cd64fc7f5d91ecf71ce1c0aef9daa348fd7a2d57555e5d77acd9d16633ca645
Jul 09 19:06:13.762084 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:13.762051    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm" event={"ID":"c36c2b9e-12bf-470e-8aa0-b14db68626cb","Type":"ContainerStarted","Data":"3cd64fc7f5d91ecf71ce1c0aef9daa348fd7a2d57555e5d77acd9d16633ca645"}
Jul 09 19:06:15.880574 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:15.880546    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"]
Jul 09 19:06:15.884491 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:15.884471    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:15.887790 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:15.887767    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hxhw9\""
Jul 09 19:06:15.904809 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:15.904780    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"]
Jul 09 19:06:16.025992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:16.025957    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l45w\" (UniqueName: \"kubernetes.io/projected/9ea2fdcf-aca4-40d4-847d-4982b3a32c15-kube-api-access-7l45w\") pod \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" (UID: \"9ea2fdcf-aca4-40d4-847d-4982b3a32c15\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:16.126504 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:16.126475    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l45w\" (UniqueName: \"kubernetes.io/projected/9ea2fdcf-aca4-40d4-847d-4982b3a32c15-kube-api-access-7l45w\") pod \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" (UID: \"9ea2fdcf-aca4-40d4-847d-4982b3a32c15\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:16.149286 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:16.149219    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l45w\" (UniqueName: \"kubernetes.io/projected/9ea2fdcf-aca4-40d4-847d-4982b3a32c15-kube-api-access-7l45w\") pod \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" (UID: \"9ea2fdcf-aca4-40d4-847d-4982b3a32c15\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:16.196366 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:16.196339    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:16.366635 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:16.366610    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"]
Jul 09 19:06:16.368443 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:06:16.368413    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea2fdcf_aca4_40d4_847d_4982b3a32c15.slice/crio-37332d0e93aeca1bd280fff08d2020e9bf0aaeea35044c365582e3be63dda649 WatchSource:0}: Error finding container 37332d0e93aeca1bd280fff08d2020e9bf0aaeea35044c365582e3be63dda649: Status 404 returned error can't find the container with id 37332d0e93aeca1bd280fff08d2020e9bf0aaeea35044c365582e3be63dda649
Jul 09 19:06:16.779237 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:16.779202    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4" event={"ID":"9ea2fdcf-aca4-40d4-847d-4982b3a32c15","Type":"ContainerStarted","Data":"37332d0e93aeca1bd280fff08d2020e9bf0aaeea35044c365582e3be63dda649"}
Jul 09 19:06:19.256799 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.256759    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"]
Jul 09 19:06:19.260404 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.260384    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"
Jul 09 19:06:19.263621 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.263602    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-8dmvx\""
Jul 09 19:06:19.274161 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.274141    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"]
Jul 09 19:06:19.355580 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.355544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blng4\" (UniqueName: \"kubernetes.io/projected/d58e8cbc-b7ed-4a2b-b125-80d72c2c6058-kube-api-access-blng4\") pod \"authorino-operator-75cc8d5cb7-zbgfv\" (UID: \"d58e8cbc-b7ed-4a2b-b125-80d72c2c6058\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"
Jul 09 19:06:19.456471 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.456442    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blng4\" (UniqueName: \"kubernetes.io/projected/d58e8cbc-b7ed-4a2b-b125-80d72c2c6058-kube-api-access-blng4\") pod \"authorino-operator-75cc8d5cb7-zbgfv\" (UID: \"d58e8cbc-b7ed-4a2b-b125-80d72c2c6058\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"
Jul 09 19:06:19.469374 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.469322    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blng4\" (UniqueName: \"kubernetes.io/projected/d58e8cbc-b7ed-4a2b-b125-80d72c2c6058-kube-api-access-blng4\") pod \"authorino-operator-75cc8d5cb7-zbgfv\" (UID: \"d58e8cbc-b7ed-4a2b-b125-80d72c2c6058\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"
Jul 09 19:06:19.575762 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.575664    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"
Jul 09 19:06:19.788800 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.788773    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"]
Jul 09 19:06:19.791385 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:06:19.791345    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58e8cbc_b7ed_4a2b_b125_80d72c2c6058.slice/crio-9a77e89dd53c5cb673e54298063dde6d8b7981f3d93327406df52d029d2af5ad WatchSource:0}: Error finding container 9a77e89dd53c5cb673e54298063dde6d8b7981f3d93327406df52d029d2af5ad: Status 404 returned error can't find the container with id 9a77e89dd53c5cb673e54298063dde6d8b7981f3d93327406df52d029d2af5ad
Jul 09 19:06:19.804627 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.804600    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm" event={"ID":"c36c2b9e-12bf-470e-8aa0-b14db68626cb","Type":"ContainerStarted","Data":"8c78d8e9575584c02845e57b370cf6303b3a2b09642ce36c11ec0df98d772a9e"}
Jul 09 19:06:19.804747 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.804714    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:19.806345 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.806322    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4" event={"ID":"9ea2fdcf-aca4-40d4-847d-4982b3a32c15","Type":"ContainerStarted","Data":"39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb"}
Jul 09 19:06:19.806451 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.806369    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:19.807579 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.807559    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv" event={"ID":"d58e8cbc-b7ed-4a2b-b125-80d72c2c6058","Type":"ContainerStarted","Data":"9a77e89dd53c5cb673e54298063dde6d8b7981f3d93327406df52d029d2af5ad"}
Jul 09 19:06:19.845671 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.845592    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm" podStartSLOduration=0.985765889 podStartE2EDuration="7.845578868s" totalImagesPullingTime="6.859812979s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:06:19.841784156 +0000 UTC m=+422.425216334" watchObservedRunningTime="2026-07-09 19:06:19.845578868 +0000 UTC m=+422.429011100"
Jul 09 19:06:19.885191 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:19.885140    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4" podStartSLOduration=1.631908119 podStartE2EDuration="4.885121973s" totalImagesPullingTime="3.253213854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:06:19.884752195 +0000 UTC m=+422.468184375" watchObservedRunningTime="2026-07-09 19:06:19.885121973 +0000 UTC m=+422.468554156"
Jul 09 19:06:20.747433 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:20.747399    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-whw9v"
Jul 09 19:06:22.838044 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:22.838011    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv" event={"ID":"d58e8cbc-b7ed-4a2b-b125-80d72c2c6058","Type":"ContainerStarted","Data":"2f221393a440940aa7511a282f3a631498e41befe7f1ab14606377de7a52e8dd"}
Jul 09 19:06:22.838557 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:22.838146    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"
Jul 09 19:06:22.859318 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:22.859274    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv" podStartSLOduration=1.340457186 podStartE2EDuration="3.859260775s" totalImagesPullingTime="2.518803589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:06:22.857094204 +0000 UTC m=+425.440526410" watchObservedRunningTime="2026-07-09 19:06:22.859260775 +0000 UTC m=+425.442692954"
Jul 09 19:06:30.827190 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:30.827154    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:30.827587 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:30.827239    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:32.498556 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.498517    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"]
Jul 09 19:06:32.503498 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.503474    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.517112 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.517089    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"]
Jul 09 19:06:32.560138 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.560110    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.560289 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.560174    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4zx6\" (UniqueName: \"kubernetes.io/projected/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-kube-api-access-q4zx6\") pod \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.660662 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.660628    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4zx6\" (UniqueName: \"kubernetes.io/projected/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-kube-api-access-q4zx6\") pod \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.660843 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.660676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.661021 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.661001    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.670246 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.670211    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4zx6\" (UniqueName: \"kubernetes.io/projected/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-kube-api-access-q4zx6\") pod \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.750649 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.750574    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"]
Jul 09 19:06:32.750965 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.750948    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:32.752815 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.752790    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"]
Jul 09 19:06:32.763195 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.763171    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"]
Jul 09 19:06:32.763452 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.763402    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" containerName="manager" containerID="cri-o://8c78d8e9575584c02845e57b370cf6303b3a2b09642ce36c11ec0df98d772a9e" gracePeriod=2
Jul 09 19:06:32.776809 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:32.776740    2569 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c36c2b9e-12bf-470e-8aa0-b14db68626cb\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\": pods \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:32.778291 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.778263    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"]
Jul 09 19:06:32.791908 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.791880    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"]
Jul 09 19:06:32.792240 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.792222    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" containerName="manager"
Jul 09 19:06:32.792240 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.792237    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" containerName="manager"
Jul 09 19:06:32.792371 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.792297    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" containerName="manager"
Jul 09 19:06:32.795904 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.795874    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"]
Jul 09 19:06:32.796022 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.796001    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:32.796087 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.796064    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" containerName="manager" containerID="cri-o://39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb" gracePeriod=2
Jul 09 19:06:32.798787 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:32.798598    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:32.801732 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.801712    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"]
Jul 09 19:06:32.808558 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.808535    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"]
Jul 09 19:06:32.830211 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:32.830176    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:32.830851 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.830830    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"]
Jul 09 19:06:32.831229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.831210    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" containerName="manager"
Jul 09 19:06:32.831229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.831229    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" containerName="manager"
Jul 09 19:06:32.831378 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.831331    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" containerName="manager"
Jul 09 19:06:32.832787 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:32.832760    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:32.834834 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.834816    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"
Jul 09 19:06:32.835480 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:32.835454    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:32.838875 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:32.838850    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:32.842082 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:32.842055    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:32.850636 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.850609    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"]
Jul 09 19:06:32.862591 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.862554    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1261102-468c-4d9c-9966-655076579487-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-slvb2\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:32.862715 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.862618    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8g6d\" (UniqueName: \"kubernetes.io/projected/9c67e2be-32aa-4fcf-be19-dae79fe74dc2-kube-api-access-z8g6d\") pod \"limitador-operator-controller-manager-859d7b78bd-cmtbn\" (UID: \"9c67e2be-32aa-4fcf-be19-dae79fe74dc2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"
Jul 09 19:06:32.862800 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.862760    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmcgz\" (UniqueName: \"kubernetes.io/projected/d1261102-468c-4d9c-9966-655076579487-kube-api-access-wmcgz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-slvb2\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:32.891333 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.891308    2569 generic.go:356] "Generic (PLEG): container finished" podID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" containerID="8c78d8e9575584c02845e57b370cf6303b3a2b09642ce36c11ec0df98d772a9e" exitCode=0
Jul 09 19:06:32.963310 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.963278    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1261102-468c-4d9c-9966-655076579487-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-slvb2\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:32.963436 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.963339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8g6d\" (UniqueName: \"kubernetes.io/projected/9c67e2be-32aa-4fcf-be19-dae79fe74dc2-kube-api-access-z8g6d\") pod \"limitador-operator-controller-manager-859d7b78bd-cmtbn\" (UID: \"9c67e2be-32aa-4fcf-be19-dae79fe74dc2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"
Jul 09 19:06:32.963436 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.963390    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmcgz\" (UniqueName: \"kubernetes.io/projected/d1261102-468c-4d9c-9966-655076579487-kube-api-access-wmcgz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-slvb2\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:32.963749 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.963670    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1261102-468c-4d9c-9966-655076579487-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-slvb2\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:32.973369 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.973339    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmcgz\" (UniqueName: \"kubernetes.io/projected/d1261102-468c-4d9c-9966-655076579487-kube-api-access-wmcgz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-slvb2\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:32.973550 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:32.973402    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8g6d\" (UniqueName: \"kubernetes.io/projected/9c67e2be-32aa-4fcf-be19-dae79fe74dc2-kube-api-access-z8g6d\") pod \"limitador-operator-controller-manager-859d7b78bd-cmtbn\" (UID: \"9c67e2be-32aa-4fcf-be19-dae79fe74dc2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"
Jul 09 19:06:33.026484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.026454    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:33.029298 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:33.029268    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:33.031551 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:33.031524    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:33.034056 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.034038    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:33.036416 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:33.036393    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:33.038520 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:33.038498    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:33.064122 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.064098    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w72l\" (UniqueName: \"kubernetes.io/projected/c36c2b9e-12bf-470e-8aa0-b14db68626cb-kube-api-access-8w72l\") pod \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") "
Jul 09 19:06:33.064193 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.064144    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36c2b9e-12bf-470e-8aa0-b14db68626cb-extensions-socket-volume\") pod \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\" (UID: \"c36c2b9e-12bf-470e-8aa0-b14db68626cb\") "
Jul 09 19:06:33.064193 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.064184    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7l45w\" (UniqueName: \"kubernetes.io/projected/9ea2fdcf-aca4-40d4-847d-4982b3a32c15-kube-api-access-7l45w\") pod \"9ea2fdcf-aca4-40d4-847d-4982b3a32c15\" (UID: \"9ea2fdcf-aca4-40d4-847d-4982b3a32c15\") "
Jul 09 19:06:33.064621 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.064599    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c36c2b9e-12bf-470e-8aa0-b14db68626cb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c36c2b9e-12bf-470e-8aa0-b14db68626cb" (UID: "c36c2b9e-12bf-470e-8aa0-b14db68626cb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:06:33.066214 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.066187    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c36c2b9e-12bf-470e-8aa0-b14db68626cb-kube-api-access-8w72l" (OuterVolumeSpecName: "kube-api-access-8w72l") pod "c36c2b9e-12bf-470e-8aa0-b14db68626cb" (UID: "c36c2b9e-12bf-470e-8aa0-b14db68626cb"). InnerVolumeSpecName "kube-api-access-8w72l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:06:33.066292 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.066257    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ea2fdcf-aca4-40d4-847d-4982b3a32c15-kube-api-access-7l45w" (OuterVolumeSpecName: "kube-api-access-7l45w") pod "9ea2fdcf-aca4-40d4-847d-4982b3a32c15" (UID: "9ea2fdcf-aca4-40d4-847d-4982b3a32c15"). InnerVolumeSpecName "kube-api-access-7l45w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:06:33.165268 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.165239    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c36c2b9e-12bf-470e-8aa0-b14db68626cb-extensions-socket-volume\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:06:33.165268 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.165262    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7l45w\" (UniqueName: \"kubernetes.io/projected/9ea2fdcf-aca4-40d4-847d-4982b3a32c15-kube-api-access-7l45w\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:06:33.165268 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.165273    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w72l\" (UniqueName: \"kubernetes.io/projected/c36c2b9e-12bf-470e-8aa0-b14db68626cb-kube-api-access-8w72l\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:06:33.187638 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.187611    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:33.193530 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.193508    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"
Jul 09 19:06:33.332039 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.332006    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"]
Jul 09 19:06:33.333281 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:06:33.333252    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1261102_468c_4d9c_9966_655076579487.slice/crio-530358e10f36b229db4d38790c6f4b3e8a8b06d0822b262aab44fa09f1152260 WatchSource:0}: Error finding container 530358e10f36b229db4d38790c6f4b3e8a8b06d0822b262aab44fa09f1152260: Status 404 returned error can't find the container with id 530358e10f36b229db4d38790c6f4b3e8a8b06d0822b262aab44fa09f1152260
Jul 09 19:06:33.355213 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.355194    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"]
Jul 09 19:06:33.356668 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:06:33.356636    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c67e2be_32aa_4fcf_be19_dae79fe74dc2.slice/crio-cfd6f00a21170037bfc705f96b3e91124dfd9deb47eb390149bd783d58bb1d6a WatchSource:0}: Error finding container cfd6f00a21170037bfc705f96b3e91124dfd9deb47eb390149bd783d58bb1d6a: Status 404 returned error can't find the container with id cfd6f00a21170037bfc705f96b3e91124dfd9deb47eb390149bd783d58bb1d6a
Jul 09 19:06:33.845817 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.845789    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-zbgfv"
Jul 09 19:06:33.877350 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.877319    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" path="/var/lib/kubelet/pods/9ea2fdcf-aca4-40d4-847d-4982b3a32c15/volumes"
Jul 09 19:06:33.877796 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.877779    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" path="/var/lib/kubelet/pods/c36c2b9e-12bf-470e-8aa0-b14db68626cb/volumes"
Jul 09 19:06:33.879800 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:33.879774    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8tkv4\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:33.882863 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:33.882830    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-8tvbm\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="c36c2b9e-12bf-470e-8aa0-b14db68626cb" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:33.899418 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.899399    2569 scope.go:117] "RemoveContainer" containerID="8c78d8e9575584c02845e57b370cf6303b3a2b09642ce36c11ec0df98d772a9e"
Jul 09 19:06:33.899574 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.899425    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-8tvbm"
Jul 09 19:06:33.902328 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.901366    2569 generic.go:356] "Generic (PLEG): container finished" podID="9ea2fdcf-aca4-40d4-847d-4982b3a32c15" containerID="39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb" exitCode=0
Jul 09 19:06:33.902328 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.901479    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8tkv4"
Jul 09 19:06:33.903831 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.903807    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn" event={"ID":"9c67e2be-32aa-4fcf-be19-dae79fe74dc2","Type":"ContainerStarted","Data":"a8ed5239bd40888396ce80b7b66e6ca742727e2740285d4495f45486780c59f7"}
Jul 09 19:06:33.903929 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.903857    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn" event={"ID":"9c67e2be-32aa-4fcf-be19-dae79fe74dc2","Type":"ContainerStarted","Data":"cfd6f00a21170037bfc705f96b3e91124dfd9deb47eb390149bd783d58bb1d6a"}
Jul 09 19:06:33.904384 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.904321    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"
Jul 09 19:06:33.906737 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.906711    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2" event={"ID":"d1261102-468c-4d9c-9966-655076579487","Type":"ContainerStarted","Data":"3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291"}
Jul 09 19:06:33.906836 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.906761    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2" event={"ID":"d1261102-468c-4d9c-9966-655076579487","Type":"ContainerStarted","Data":"530358e10f36b229db4d38790c6f4b3e8a8b06d0822b262aab44fa09f1152260"}
Jul 09 19:06:33.906836 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.906790    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:33.912463 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.912442    2569 scope.go:117] "RemoveContainer" containerID="39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb"
Jul 09 19:06:33.920779 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.920762    2569 scope.go:117] "RemoveContainer" containerID="39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb"
Jul 09 19:06:33.921124 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:33.921103    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb\": container with ID starting with 39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb not found: ID does not exist" containerID="39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb"
Jul 09 19:06:33.921188 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.921133    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb"} err="failed to get container status \"39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb\": rpc error: code = NotFound desc = could not find container \"39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb\": container with ID starting with 39da847b29b5ca7aef32c4deca882b2971d31ee60d5a6fcf8d1163b964a3f2eb not found: ID does not exist"
Jul 09 19:06:33.925769 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.925733    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn" podStartSLOduration=1.9257223209999998 podStartE2EDuration="1.925722321s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:06:33.92348333 +0000 UTC m=+436.506915510" watchObservedRunningTime="2026-07-09 19:06:33.925722321 +0000 UTC m=+436.509154544"
Jul 09 19:06:33.953326 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:33.953253    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2" podStartSLOduration=1.9532425679999998 podStartE2EDuration="1.953242568s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:06:33.952622955 +0000 UTC m=+436.536055149" watchObservedRunningTime="2026-07-09 19:06:33.953242568 +0000 UTC m=+436.536674747"
Jul 09 19:06:35.296941 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:35.296896    2569 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-d5d7x_kuadrant-system_fe7ae588-9ab4-45fe-9ba0-115ae6c0498f_0(cd771506122e20b42966fdb6ce39932071ca35a3cf9f63683ed04fda5024b5d6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-d5d7x to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}"
Jul 09 19:06:35.297181 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:35.296974    2569 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-d5d7x_kuadrant-system_fe7ae588-9ab4-45fe-9ba0-115ae6c0498f_0(cd771506122e20b42966fdb6ce39932071ca35a3cf9f63683ed04fda5024b5d6): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-d5d7x to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:35.918356 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:35.918322    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:35.926366 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:35.926345    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:35.929235 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:35.929214    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="fe7ae588-9ab4-45fe-9ba0-115ae6c0498f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:35.987976 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:35.987952    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-extensions-socket-volume\") pod \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") "
Jul 09 19:06:35.988104 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:35.987996    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4zx6\" (UniqueName: \"kubernetes.io/projected/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-kube-api-access-q4zx6\") pod \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\" (UID: \"fe7ae588-9ab4-45fe-9ba0-115ae6c0498f\") "
Jul 09 19:06:35.988251 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:35.988219    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fe7ae588-9ab4-45fe-9ba0-115ae6c0498f" (UID: "fe7ae588-9ab4-45fe-9ba0-115ae6c0498f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:06:35.990243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:35.990220    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-kube-api-access-q4zx6" (OuterVolumeSpecName: "kube-api-access-q4zx6") pod "fe7ae588-9ab4-45fe-9ba0-115ae6c0498f" (UID: "fe7ae588-9ab4-45fe-9ba0-115ae6c0498f"). InnerVolumeSpecName "kube-api-access-q4zx6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:06:36.088997 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:36.088966    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-extensions-socket-volume\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:06:36.088997 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:36.088991    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4zx6\" (UniqueName: \"kubernetes.io/projected/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f-kube-api-access-q4zx6\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:06:36.923646 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:36.923617    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:36.927026 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:36.926999    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="fe7ae588-9ab4-45fe-9ba0-115ae6c0498f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:36.932152 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:36.932125    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="fe7ae588-9ab4-45fe-9ba0-115ae6c0498f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:37.877000 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:06:37.876967    2569 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-d5d7x\" is forbidden: User \"system:node:ip-10-0-128-157.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-157.ec2.internal' and this object" podUID="fe7ae588-9ab4-45fe-9ba0-115ae6c0498f" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-d5d7x"
Jul 09 19:06:37.877653 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:37.877628    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe7ae588-9ab4-45fe-9ba0-115ae6c0498f" path="/var/lib/kubelet/pods/fe7ae588-9ab4-45fe-9ba0-115ae6c0498f/volumes"
Jul 09 19:06:44.916006 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:44.915970    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:06:45.921039 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:06:45.921015    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-cmtbn"
Jul 09 19:07:01.720958 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:01.720927    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"]
Jul 09 19:07:01.721359 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:01.721154    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2" podUID="d1261102-468c-4d9c-9966-655076579487" containerName="manager" containerID="cri-o://3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291" gracePeriod=10
Jul 09 19:07:01.979111 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:01.979043    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:07:02.065826 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.065787    2569 generic.go:356] "Generic (PLEG): container finished" podID="d1261102-468c-4d9c-9966-655076579487" containerID="3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291" exitCode=0
Jul 09 19:07:02.066012 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.065881    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"
Jul 09 19:07:02.066012 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.065927    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2" event={"ID":"d1261102-468c-4d9c-9966-655076579487","Type":"ContainerDied","Data":"3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291"}
Jul 09 19:07:02.066012 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.065985    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2" event={"ID":"d1261102-468c-4d9c-9966-655076579487","Type":"ContainerDied","Data":"530358e10f36b229db4d38790c6f4b3e8a8b06d0822b262aab44fa09f1152260"}
Jul 09 19:07:02.066012 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.066013    2569 scope.go:117] "RemoveContainer" containerID="3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291"
Jul 09 19:07:02.075451 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.075426    2569 scope.go:117] "RemoveContainer" containerID="3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291"
Jul 09 19:07:02.075740 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:07:02.075716    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291\": container with ID starting with 3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291 not found: ID does not exist" containerID="3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291"
Jul 09 19:07:02.075820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.075751    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291"} err="failed to get container status \"3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291\": rpc error: code = NotFound desc = could not find container \"3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291\": container with ID starting with 3dd64d4406c00eca6be182d44f27b05d1058a15cf0cd060eceb9a753f19c6291 not found: ID does not exist"
Jul 09 19:07:02.077443 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.077425    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"]
Jul 09 19:07:02.077807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.077794    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1261102-468c-4d9c-9966-655076579487" containerName="manager"
Jul 09 19:07:02.077848 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.077810    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1261102-468c-4d9c-9966-655076579487" containerName="manager"
Jul 09 19:07:02.077884 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.077867    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1261102-468c-4d9c-9966-655076579487" containerName="manager"
Jul 09 19:07:02.080594 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.080578    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.095407 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.095385    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"]
Jul 09 19:07:02.118752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.118729    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmcgz\" (UniqueName: \"kubernetes.io/projected/d1261102-468c-4d9c-9966-655076579487-kube-api-access-wmcgz\") pod \"d1261102-468c-4d9c-9966-655076579487\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") "
Jul 09 19:07:02.118863 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.118805    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1261102-468c-4d9c-9966-655076579487-extensions-socket-volume\") pod \"d1261102-468c-4d9c-9966-655076579487\" (UID: \"d1261102-468c-4d9c-9966-655076579487\") "
Jul 09 19:07:02.119176 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.119157    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1261102-468c-4d9c-9966-655076579487-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d1261102-468c-4d9c-9966-655076579487" (UID: "d1261102-468c-4d9c-9966-655076579487"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:07:02.120950 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.120920    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1261102-468c-4d9c-9966-655076579487-kube-api-access-wmcgz" (OuterVolumeSpecName: "kube-api-access-wmcgz") pod "d1261102-468c-4d9c-9966-655076579487" (UID: "d1261102-468c-4d9c-9966-655076579487"). InnerVolumeSpecName "kube-api-access-wmcgz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:02.220302 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.220269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/141e8fc2-c7a1-4574-a413-772ab1554519-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5rw94\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.220479 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.220339    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dt4w\" (UniqueName: \"kubernetes.io/projected/141e8fc2-c7a1-4574-a413-772ab1554519-kube-api-access-9dt4w\") pod \"kuadrant-operator-controller-manager-86878c769f-5rw94\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.220479 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.220402    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d1261102-468c-4d9c-9966-655076579487-extensions-socket-volume\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:07:02.220479 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.220421    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmcgz\" (UniqueName: \"kubernetes.io/projected/d1261102-468c-4d9c-9966-655076579487-kube-api-access-wmcgz\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:07:02.321591 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.321503    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dt4w\" (UniqueName: \"kubernetes.io/projected/141e8fc2-c7a1-4574-a413-772ab1554519-kube-api-access-9dt4w\") pod \"kuadrant-operator-controller-manager-86878c769f-5rw94\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.321591 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.321560    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/141e8fc2-c7a1-4574-a413-772ab1554519-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5rw94\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.321909 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.321893    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/141e8fc2-c7a1-4574-a413-772ab1554519-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-5rw94\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.331305 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.331284    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dt4w\" (UniqueName: \"kubernetes.io/projected/141e8fc2-c7a1-4574-a413-772ab1554519-kube-api-access-9dt4w\") pod \"kuadrant-operator-controller-manager-86878c769f-5rw94\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.390231 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.390200    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"]
Jul 09 19:07:02.390534 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.390518    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:02.398422 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.398395    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-slvb2"]
Jul 09 19:07:02.528855 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:02.528829    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"]
Jul 09 19:07:02.531243 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:07:02.531221    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod141e8fc2_c7a1_4574_a413_772ab1554519.slice/crio-3604b086e53b220c37f57b0042972ad294e3a65675f68e5d7ab33c018c7e1de6 WatchSource:0}: Error finding container 3604b086e53b220c37f57b0042972ad294e3a65675f68e5d7ab33c018c7e1de6: Status 404 returned error can't find the container with id 3604b086e53b220c37f57b0042972ad294e3a65675f68e5d7ab33c018c7e1de6
Jul 09 19:07:03.072968 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:03.072932    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94" event={"ID":"141e8fc2-c7a1-4574-a413-772ab1554519","Type":"ContainerStarted","Data":"0ca0b8980eb163c2b9777093f10d9fd4a1e199629c8e1090f8f3565a3ebf1ad3"}
Jul 09 19:07:03.073436 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:03.072997    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:03.073436 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:03.073018    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94" event={"ID":"141e8fc2-c7a1-4574-a413-772ab1554519","Type":"ContainerStarted","Data":"3604b086e53b220c37f57b0042972ad294e3a65675f68e5d7ab33c018c7e1de6"}
Jul 09 19:07:03.102078 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:03.102027    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94" podStartSLOduration=1.102008678 podStartE2EDuration="1.102008678s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:03.099973766 +0000 UTC m=+465.683405968" watchObservedRunningTime="2026-07-09 19:07:03.102008678 +0000 UTC m=+465.685440858"
Jul 09 19:07:03.877695 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:03.877648    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d1261102-468c-4d9c-9966-655076579487" path="/var/lib/kubelet/pods/d1261102-468c-4d9c-9966-655076579487/volumes"
Jul 09 19:07:14.082236 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:14.082197    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:07:19.738887 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.738849    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"]
Jul 09 19:07:19.742108 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.742085    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:19.745435 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.745411    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-njbf7\""
Jul 09 19:07:19.745536 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.745450    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 09 19:07:19.751852 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.751830    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"]
Jul 09 19:07:19.769307 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.769276    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p95c2\" (UniqueName: \"kubernetes.io/projected/364fcf79-5a7c-4eac-ba77-79965a2d971e-kube-api-access-p95c2\") pod \"limitador-limitador-64c94fd64f-cgv8q\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:19.769307 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.769309    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/364fcf79-5a7c-4eac-ba77-79965a2d971e-config-file\") pod \"limitador-limitador-64c94fd64f-cgv8q\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:19.833978 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.833945    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"]
Jul 09 19:07:19.869747 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.869719    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p95c2\" (UniqueName: \"kubernetes.io/projected/364fcf79-5a7c-4eac-ba77-79965a2d971e-kube-api-access-p95c2\") pod \"limitador-limitador-64c94fd64f-cgv8q\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:19.869893 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.869753    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/364fcf79-5a7c-4eac-ba77-79965a2d971e-config-file\") pod \"limitador-limitador-64c94fd64f-cgv8q\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:19.870420 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.870398    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/364fcf79-5a7c-4eac-ba77-79965a2d971e-config-file\") pod \"limitador-limitador-64c94fd64f-cgv8q\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:19.878734 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:19.878713    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p95c2\" (UniqueName: \"kubernetes.io/projected/364fcf79-5a7c-4eac-ba77-79965a2d971e-kube-api-access-p95c2\") pod \"limitador-limitador-64c94fd64f-cgv8q\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:20.053040 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.052949    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:20.196206 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.196180    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"]
Jul 09 19:07:20.197545 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:07:20.197517    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364fcf79_5a7c_4eac_ba77_79965a2d971e.slice/crio-3cd21aca4895db41a497ab78c695f9d0c79f738164b5957af54350af8384500f WatchSource:0}: Error finding container 3cd21aca4895db41a497ab78c695f9d0c79f738164b5957af54350af8384500f: Status 404 returned error can't find the container with id 3cd21aca4895db41a497ab78c695f9d0c79f738164b5957af54350af8384500f
Jul 09 19:07:20.581468 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.581433    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-f2ggj"]
Jul 09 19:07:20.584352 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.584333    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f2ggj"
Jul 09 19:07:20.589891 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.589867    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fcl7g\""
Jul 09 19:07:20.627331 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.627304    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f2ggj"]
Jul 09 19:07:20.675752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.675719    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fkf8\" (UniqueName: \"kubernetes.io/projected/c4206e8f-32a4-442f-98c3-c32b60c3b743-kube-api-access-2fkf8\") pod \"authorino-58cd955675-f2ggj\" (UID: \"c4206e8f-32a4-442f-98c3-c32b60c3b743\") " pod="kuadrant-system/authorino-58cd955675-f2ggj"
Jul 09 19:07:20.776170 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.776140    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fkf8\" (UniqueName: \"kubernetes.io/projected/c4206e8f-32a4-442f-98c3-c32b60c3b743-kube-api-access-2fkf8\") pod \"authorino-58cd955675-f2ggj\" (UID: \"c4206e8f-32a4-442f-98c3-c32b60c3b743\") " pod="kuadrant-system/authorino-58cd955675-f2ggj"
Jul 09 19:07:20.790899 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.790871    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fkf8\" (UniqueName: \"kubernetes.io/projected/c4206e8f-32a4-442f-98c3-c32b60c3b743-kube-api-access-2fkf8\") pod \"authorino-58cd955675-f2ggj\" (UID: \"c4206e8f-32a4-442f-98c3-c32b60c3b743\") " pod="kuadrant-system/authorino-58cd955675-f2ggj"
Jul 09 19:07:20.893422 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:20.893289    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f2ggj"
Jul 09 19:07:21.090092 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:21.090062    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-f2ggj"]
Jul 09 19:07:21.092339 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:07:21.092299    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4206e8f_32a4_442f_98c3_c32b60c3b743.slice/crio-f0e74e04b94b1c789e5712c586daa0c8827396b189e81c28c3f632149c975d07 WatchSource:0}: Error finding container f0e74e04b94b1c789e5712c586daa0c8827396b189e81c28c3f632149c975d07: Status 404 returned error can't find the container with id f0e74e04b94b1c789e5712c586daa0c8827396b189e81c28c3f632149c975d07
Jul 09 19:07:21.179743 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:21.179645    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f2ggj" event={"ID":"c4206e8f-32a4-442f-98c3-c32b60c3b743","Type":"ContainerStarted","Data":"f0e74e04b94b1c789e5712c586daa0c8827396b189e81c28c3f632149c975d07"}
Jul 09 19:07:21.181136 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:21.181105    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q" event={"ID":"364fcf79-5a7c-4eac-ba77-79965a2d971e","Type":"ContainerStarted","Data":"3cd21aca4895db41a497ab78c695f9d0c79f738164b5957af54350af8384500f"}
Jul 09 19:07:25.219608 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:25.219572    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f2ggj" event={"ID":"c4206e8f-32a4-442f-98c3-c32b60c3b743","Type":"ContainerStarted","Data":"9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a"}
Jul 09 19:07:25.221102 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:25.221079    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q" event={"ID":"364fcf79-5a7c-4eac-ba77-79965a2d971e","Type":"ContainerStarted","Data":"473080bb4e7f54df060d1acd5f0e43077181cc95912ac4a04c71d78113c0e347"}
Jul 09 19:07:25.221217 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:25.221146    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:25.237491 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:25.237449    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-f2ggj" podStartSLOduration=1.508923855 podStartE2EDuration="5.237436029s" totalImagesPullingTime="3.728512174s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:25.237090775 +0000 UTC m=+487.820522955" watchObservedRunningTime="2026-07-09 19:07:25.237436029 +0000 UTC m=+487.820868212"
Jul 09 19:07:25.257484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:25.257440    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q" podStartSLOduration=1.671956426 podStartE2EDuration="6.257425229s" totalImagesPullingTime="4.585468803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:25.256195546 +0000 UTC m=+487.839627726" watchObservedRunningTime="2026-07-09 19:07:25.257425229 +0000 UTC m=+487.840857408"
Jul 09 19:07:26.002677 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:26.002635    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f2ggj"]
Jul 09 19:07:27.231521 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:27.231481    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-f2ggj" podUID="c4206e8f-32a4-442f-98c3-c32b60c3b743" containerName="authorino" containerID="cri-o://9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a" gracePeriod=30
Jul 09 19:07:27.474698 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:27.474668    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f2ggj"
Jul 09 19:07:27.534381 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:27.534297    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fkf8\" (UniqueName: \"kubernetes.io/projected/c4206e8f-32a4-442f-98c3-c32b60c3b743-kube-api-access-2fkf8\") pod \"c4206e8f-32a4-442f-98c3-c32b60c3b743\" (UID: \"c4206e8f-32a4-442f-98c3-c32b60c3b743\") "
Jul 09 19:07:27.536521 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:27.536489    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4206e8f-32a4-442f-98c3-c32b60c3b743-kube-api-access-2fkf8" (OuterVolumeSpecName: "kube-api-access-2fkf8") pod "c4206e8f-32a4-442f-98c3-c32b60c3b743" (UID: "c4206e8f-32a4-442f-98c3-c32b60c3b743"). InnerVolumeSpecName "kube-api-access-2fkf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:27.635016 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:27.634985    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fkf8\" (UniqueName: \"kubernetes.io/projected/c4206e8f-32a4-442f-98c3-c32b60c3b743-kube-api-access-2fkf8\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:07:28.237514 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.237422    2569 generic.go:356] "Generic (PLEG): container finished" podID="c4206e8f-32a4-442f-98c3-c32b60c3b743" containerID="9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a" exitCode=0
Jul 09 19:07:28.237514 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.237464    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-f2ggj"
Jul 09 19:07:28.238026 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.237514    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f2ggj" event={"ID":"c4206e8f-32a4-442f-98c3-c32b60c3b743","Type":"ContainerDied","Data":"9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a"}
Jul 09 19:07:28.238026 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.237558    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-f2ggj" event={"ID":"c4206e8f-32a4-442f-98c3-c32b60c3b743","Type":"ContainerDied","Data":"f0e74e04b94b1c789e5712c586daa0c8827396b189e81c28c3f632149c975d07"}
Jul 09 19:07:28.238026 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.237576    2569 scope.go:117] "RemoveContainer" containerID="9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a"
Jul 09 19:07:28.247017 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.247000    2569 scope.go:117] "RemoveContainer" containerID="9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a"
Jul 09 19:07:28.247275 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:07:28.247257    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a\": container with ID starting with 9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a not found: ID does not exist" containerID="9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a"
Jul 09 19:07:28.247328 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.247283    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a"} err="failed to get container status \"9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a\": rpc error: code = NotFound desc = could not find container \"9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a\": container with ID starting with 9adc048bba560f89bcb918f8f3c179435ee8c2d1f19f9830480cacbe79293b8a not found: ID does not exist"
Jul 09 19:07:28.257311 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.257284    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-f2ggj"]
Jul 09 19:07:28.260991 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:28.260969    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-f2ggj"]
Jul 09 19:07:29.877471 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:29.877426    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4206e8f-32a4-442f-98c3-c32b60c3b743" path="/var/lib/kubelet/pods/c4206e8f-32a4-442f-98c3-c32b60c3b743/volumes"
Jul 09 19:07:36.226827 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:36.226798    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:36.883233 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:36.883197    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"]
Jul 09 19:07:36.883418 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:36.883388    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q" podUID="364fcf79-5a7c-4eac-ba77-79965a2d971e" containerName="limitador" containerID="cri-o://473080bb4e7f54df060d1acd5f0e43077181cc95912ac4a04c71d78113c0e347" gracePeriod=30
Jul 09 19:07:37.286803 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:37.286767    2569 generic.go:356] "Generic (PLEG): container finished" podID="364fcf79-5a7c-4eac-ba77-79965a2d971e" containerID="473080bb4e7f54df060d1acd5f0e43077181cc95912ac4a04c71d78113c0e347" exitCode=0
Jul 09 19:07:37.287153 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:37.286849    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q" event={"ID":"364fcf79-5a7c-4eac-ba77-79965a2d971e","Type":"ContainerDied","Data":"473080bb4e7f54df060d1acd5f0e43077181cc95912ac4a04c71d78113c0e347"}
Jul 09 19:07:37.834292 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:37.834271    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:37.916357 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:37.916326    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p95c2\" (UniqueName: \"kubernetes.io/projected/364fcf79-5a7c-4eac-ba77-79965a2d971e-kube-api-access-p95c2\") pod \"364fcf79-5a7c-4eac-ba77-79965a2d971e\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") "
Jul 09 19:07:37.916488 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:37.916394    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/364fcf79-5a7c-4eac-ba77-79965a2d971e-config-file\") pod \"364fcf79-5a7c-4eac-ba77-79965a2d971e\" (UID: \"364fcf79-5a7c-4eac-ba77-79965a2d971e\") "
Jul 09 19:07:37.916773 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:37.916744    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/364fcf79-5a7c-4eac-ba77-79965a2d971e-config-file" (OuterVolumeSpecName: "config-file") pod "364fcf79-5a7c-4eac-ba77-79965a2d971e" (UID: "364fcf79-5a7c-4eac-ba77-79965a2d971e"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 09 19:07:37.918478 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:37.918458    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/364fcf79-5a7c-4eac-ba77-79965a2d971e-kube-api-access-p95c2" (OuterVolumeSpecName: "kube-api-access-p95c2") pod "364fcf79-5a7c-4eac-ba77-79965a2d971e" (UID: "364fcf79-5a7c-4eac-ba77-79965a2d971e"). InnerVolumeSpecName "kube-api-access-p95c2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:38.017638 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:38.017558    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p95c2\" (UniqueName: \"kubernetes.io/projected/364fcf79-5a7c-4eac-ba77-79965a2d971e-kube-api-access-p95c2\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:07:38.017638 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:38.017585    2569 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/364fcf79-5a7c-4eac-ba77-79965a2d971e-config-file\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:07:38.293572 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:38.293485    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q" event={"ID":"364fcf79-5a7c-4eac-ba77-79965a2d971e","Type":"ContainerDied","Data":"3cd21aca4895db41a497ab78c695f9d0c79f738164b5957af54350af8384500f"}
Jul 09 19:07:38.293572 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:38.293553    2569 scope.go:117] "RemoveContainer" containerID="473080bb4e7f54df060d1acd5f0e43077181cc95912ac4a04c71d78113c0e347"
Jul 09 19:07:38.294039 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:38.293493    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"
Jul 09 19:07:38.315894 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:38.315863    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"]
Jul 09 19:07:38.319541 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:38.319520    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-cgv8q"]
Jul 09 19:07:39.877539 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:39.877495    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="364fcf79-5a7c-4eac-ba77-79965a2d971e" path="/var/lib/kubelet/pods/364fcf79-5a7c-4eac-ba77-79965a2d971e/volumes"
Jul 09 19:07:41.403443 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.403408    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"]
Jul 09 19:07:41.403807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.403781    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4206e8f-32a4-442f-98c3-c32b60c3b743" containerName="authorino"
Jul 09 19:07:41.403807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.403793    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4206e8f-32a4-442f-98c3-c32b60c3b743" containerName="authorino"
Jul 09 19:07:41.403884 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.403813    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="364fcf79-5a7c-4eac-ba77-79965a2d971e" containerName="limitador"
Jul 09 19:07:41.403884 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.403819    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="364fcf79-5a7c-4eac-ba77-79965a2d971e" containerName="limitador"
Jul 09 19:07:41.403884 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.403872    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4206e8f-32a4-442f-98c3-c32b60c3b743" containerName="authorino"
Jul 09 19:07:41.403884 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.403882    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="364fcf79-5a7c-4eac-ba77-79965a2d971e" containerName="limitador"
Jul 09 19:07:41.405977 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.405962    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.409810 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.409790    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 09 19:07:41.409940 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.409871    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 09 19:07:41.410086 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.410073    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-k2vlw\""
Jul 09 19:07:41.411287 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.411272    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 09 19:07:41.418129 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.418105    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"]
Jul 09 19:07:41.444015 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.443989    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/88e1dc59-5cd2-4cae-852e-4ded32e4261e-data\") pod \"postgres-69fb4594cf-6gjf9\" (UID: \"88e1dc59-5cd2-4cae-852e-4ded32e4261e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.444131 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.444030    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gwwx2\" (UniqueName: \"kubernetes.io/projected/88e1dc59-5cd2-4cae-852e-4ded32e4261e-kube-api-access-gwwx2\") pod \"postgres-69fb4594cf-6gjf9\" (UID: \"88e1dc59-5cd2-4cae-852e-4ded32e4261e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.544366 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.544339    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gwwx2\" (UniqueName: \"kubernetes.io/projected/88e1dc59-5cd2-4cae-852e-4ded32e4261e-kube-api-access-gwwx2\") pod \"postgres-69fb4594cf-6gjf9\" (UID: \"88e1dc59-5cd2-4cae-852e-4ded32e4261e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.544476 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.544403    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/88e1dc59-5cd2-4cae-852e-4ded32e4261e-data\") pod \"postgres-69fb4594cf-6gjf9\" (UID: \"88e1dc59-5cd2-4cae-852e-4ded32e4261e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.544723 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.544708    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/88e1dc59-5cd2-4cae-852e-4ded32e4261e-data\") pod \"postgres-69fb4594cf-6gjf9\" (UID: \"88e1dc59-5cd2-4cae-852e-4ded32e4261e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.554699 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.554662    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gwwx2\" (UniqueName: \"kubernetes.io/projected/88e1dc59-5cd2-4cae-852e-4ded32e4261e-kube-api-access-gwwx2\") pod \"postgres-69fb4594cf-6gjf9\" (UID: \"88e1dc59-5cd2-4cae-852e-4ded32e4261e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.717422 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.717351    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:41.849972 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:41.849940    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"]
Jul 09 19:07:41.851546 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:07:41.851513    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e1dc59_5cd2_4cae_852e_4ded32e4261e.slice/crio-aea54e9a29e226cf6d1602eb894cfc508259592f4259f06f90c8f82904741368 WatchSource:0}: Error finding container aea54e9a29e226cf6d1602eb894cfc508259592f4259f06f90c8f82904741368: Status 404 returned error can't find the container with id aea54e9a29e226cf6d1602eb894cfc508259592f4259f06f90c8f82904741368
Jul 09 19:07:42.317606 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:42.317569    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9" event={"ID":"88e1dc59-5cd2-4cae-852e-4ded32e4261e","Type":"ContainerStarted","Data":"aea54e9a29e226cf6d1602eb894cfc508259592f4259f06f90c8f82904741368"}
Jul 09 19:07:47.351698 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:47.351658    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9" event={"ID":"88e1dc59-5cd2-4cae-852e-4ded32e4261e","Type":"ContainerStarted","Data":"85782b892d17630bfbfd83556a8912e9edf39b1e66a03cacb04bcce0bb269af4"}
Jul 09 19:07:47.352060 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:47.351791    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:53.387511 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:53.387472    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9"
Jul 09 19:07:53.405465 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:53.405420    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6gjf9" podStartSLOduration=7.111360688 podStartE2EDuration="12.405406245s" totalImagesPullingTime="5.294045557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:47.367788865 +0000 UTC m=+509.951221045" watchObservedRunningTime="2026-07-09 19:07:53.405406245 +0000 UTC m=+515.988838424"
Jul 09 19:07:53.921035 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:53.921003    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5r2b"]
Jul 09 19:07:53.927558 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:53.927538    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5r2b"
Jul 09 19:07:53.930499 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:53.930477    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fcl7g\""
Jul 09 19:07:53.930850 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:53.930830    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5r2b"]
Jul 09 19:07:54.055334 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.055303    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jcqd\" (UniqueName: \"kubernetes.io/projected/22b9cc4c-c683-4e9b-90c2-5b59b80cf851-kube-api-access-7jcqd\") pod \"authorino-5d4b67c446-t5r2b\" (UID: \"22b9cc4c-c683-4e9b-90c2-5b59b80cf851\") " pod="kuadrant-system/authorino-5d4b67c446-t5r2b"
Jul 09 19:07:54.136581 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.136546    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5r2b"]
Jul 09 19:07:54.136825 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:07:54.136806    2569 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-7jcqd], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-t5r2b" podUID="22b9cc4c-c683-4e9b-90c2-5b59b80cf851"
Jul 09 19:07:54.156553 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.156526    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jcqd\" (UniqueName: \"kubernetes.io/projected/22b9cc4c-c683-4e9b-90c2-5b59b80cf851-kube-api-access-7jcqd\") pod \"authorino-5d4b67c446-t5r2b\" (UID: \"22b9cc4c-c683-4e9b-90c2-5b59b80cf851\") " pod="kuadrant-system/authorino-5d4b67c446-t5r2b"
Jul 09 19:07:54.162326 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.162305    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5b899f7f57-s5p9r"]
Jul 09 19:07:54.166255 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.166237    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b899f7f57-s5p9r"
Jul 09 19:07:54.171718 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.171651    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5b899f7f57-s5p9r"]
Jul 09 19:07:54.173229 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.173184    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jcqd\" (UniqueName: \"kubernetes.io/projected/22b9cc4c-c683-4e9b-90c2-5b59b80cf851-kube-api-access-7jcqd\") pod \"authorino-5d4b67c446-t5r2b\" (UID: \"22b9cc4c-c683-4e9b-90c2-5b59b80cf851\") " pod="kuadrant-system/authorino-5d4b67c446-t5r2b"
Jul 09 19:07:54.257613 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.257581    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qmb7\" (UniqueName: \"kubernetes.io/projected/509d2dae-f425-4cc4-b548-2e56ce22f2fa-kube-api-access-5qmb7\") pod \"authorino-5b899f7f57-s5p9r\" (UID: \"509d2dae-f425-4cc4-b548-2e56ce22f2fa\") " pod="kuadrant-system/authorino-5b899f7f57-s5p9r"
Jul 09 19:07:54.358723 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.358695    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qmb7\" (UniqueName: \"kubernetes.io/projected/509d2dae-f425-4cc4-b548-2e56ce22f2fa-kube-api-access-5qmb7\") pod \"authorino-5b899f7f57-s5p9r\" (UID: \"509d2dae-f425-4cc4-b548-2e56ce22f2fa\") " pod="kuadrant-system/authorino-5b899f7f57-s5p9r"
Jul 09 19:07:54.368160 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.368135    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qmb7\" (UniqueName: \"kubernetes.io/projected/509d2dae-f425-4cc4-b548-2e56ce22f2fa-kube-api-access-5qmb7\") pod \"authorino-5b899f7f57-s5p9r\" (UID: \"509d2dae-f425-4cc4-b548-2e56ce22f2fa\") " pod="kuadrant-system/authorino-5b899f7f57-s5p9r"
Jul 09 19:07:54.371734 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.371710    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b899f7f57-s5p9r"]
Jul 09 19:07:54.371917 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.371906    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b899f7f57-s5p9r"
Jul 09 19:07:54.387525 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.387500    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5r2b"
Jul 09 19:07:54.403970 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.403950    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5r2b"
Jul 09 19:07:54.459421 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.459380    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jcqd\" (UniqueName: \"kubernetes.io/projected/22b9cc4c-c683-4e9b-90c2-5b59b80cf851-kube-api-access-7jcqd\") pod \"22b9cc4c-c683-4e9b-90c2-5b59b80cf851\" (UID: \"22b9cc4c-c683-4e9b-90c2-5b59b80cf851\") "
Jul 09 19:07:54.462092 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.462065    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22b9cc4c-c683-4e9b-90c2-5b59b80cf851-kube-api-access-7jcqd" (OuterVolumeSpecName: "kube-api-access-7jcqd") pod "22b9cc4c-c683-4e9b-90c2-5b59b80cf851" (UID: "22b9cc4c-c683-4e9b-90c2-5b59b80cf851"). InnerVolumeSpecName "kube-api-access-7jcqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:54.515841 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.515812    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b899f7f57-s5p9r"]
Jul 09 19:07:54.516743 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:07:54.516718    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod509d2dae_f425_4cc4_b548_2e56ce22f2fa.slice/crio-9fc4760f3ae482aa3c5b97db7b943ab0011bf9fbf1d81e1a1282332978e4bbb3 WatchSource:0}: Error finding container 9fc4760f3ae482aa3c5b97db7b943ab0011bf9fbf1d81e1a1282332978e4bbb3: Status 404 returned error can't find the container with id 9fc4760f3ae482aa3c5b97db7b943ab0011bf9fbf1d81e1a1282332978e4bbb3
Jul 09 19:07:54.560671 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:54.560640    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jcqd\" (UniqueName: \"kubernetes.io/projected/22b9cc4c-c683-4e9b-90c2-5b59b80cf851-kube-api-access-7jcqd\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:07:55.394567 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.394534    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5b899f7f57-s5p9r" podUID="509d2dae-f425-4cc4-b548-2e56ce22f2fa" containerName="authorino" containerID="cri-o://4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925" gracePeriod=30
Jul 09 19:07:55.394567 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.394559    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-t5r2b"
Jul 09 19:07:55.395371 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.394539    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b899f7f57-s5p9r" event={"ID":"509d2dae-f425-4cc4-b548-2e56ce22f2fa","Type":"ContainerStarted","Data":"4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925"}
Jul 09 19:07:55.395371 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.394660    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b899f7f57-s5p9r" event={"ID":"509d2dae-f425-4cc4-b548-2e56ce22f2fa","Type":"ContainerStarted","Data":"9fc4760f3ae482aa3c5b97db7b943ab0011bf9fbf1d81e1a1282332978e4bbb3"}
Jul 09 19:07:55.410262 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.410213    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5b899f7f57-s5p9r" podStartSLOduration=1.049461529 podStartE2EDuration="1.410196812s" totalImagesPullingTime="360.735283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:07:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:07:55.409003894 +0000 UTC m=+517.992436095" watchObservedRunningTime="2026-07-09 19:07:55.410196812 +0000 UTC m=+517.993628993"
Jul 09 19:07:55.434458 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.434428    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5r2b"]
Jul 09 19:07:55.436615 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.436595    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-t5r2b"]
Jul 09 19:07:55.658195 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.658142    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b899f7f57-s5p9r"
Jul 09 19:07:55.770933 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.770896    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qmb7\" (UniqueName: \"kubernetes.io/projected/509d2dae-f425-4cc4-b548-2e56ce22f2fa-kube-api-access-5qmb7\") pod \"509d2dae-f425-4cc4-b548-2e56ce22f2fa\" (UID: \"509d2dae-f425-4cc4-b548-2e56ce22f2fa\") "
Jul 09 19:07:55.773077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.773053    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/509d2dae-f425-4cc4-b548-2e56ce22f2fa-kube-api-access-5qmb7" (OuterVolumeSpecName: "kube-api-access-5qmb7") pod "509d2dae-f425-4cc4-b548-2e56ce22f2fa" (UID: "509d2dae-f425-4cc4-b548-2e56ce22f2fa"). InnerVolumeSpecName "kube-api-access-5qmb7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:07:55.872201 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.872168    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qmb7\" (UniqueName: \"kubernetes.io/projected/509d2dae-f425-4cc4-b548-2e56ce22f2fa-kube-api-access-5qmb7\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:07:55.877971 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:55.877940    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22b9cc4c-c683-4e9b-90c2-5b59b80cf851" path="/var/lib/kubelet/pods/22b9cc4c-c683-4e9b-90c2-5b59b80cf851/volumes"
Jul 09 19:07:56.403421 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.403386    2569 generic.go:356] "Generic (PLEG): container finished" podID="509d2dae-f425-4cc4-b548-2e56ce22f2fa" containerID="4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925" exitCode=0
Jul 09 19:07:56.403810 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.403444    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5b899f7f57-s5p9r"
Jul 09 19:07:56.403810 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.403473    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b899f7f57-s5p9r" event={"ID":"509d2dae-f425-4cc4-b548-2e56ce22f2fa","Type":"ContainerDied","Data":"4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925"}
Jul 09 19:07:56.403810 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.403515    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5b899f7f57-s5p9r" event={"ID":"509d2dae-f425-4cc4-b548-2e56ce22f2fa","Type":"ContainerDied","Data":"9fc4760f3ae482aa3c5b97db7b943ab0011bf9fbf1d81e1a1282332978e4bbb3"}
Jul 09 19:07:56.403810 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.403532    2569 scope.go:117] "RemoveContainer" containerID="4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925"
Jul 09 19:07:56.412963 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.412945    2569 scope.go:117] "RemoveContainer" containerID="4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925"
Jul 09 19:07:56.413199 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:07:56.413180    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925\": container with ID starting with 4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925 not found: ID does not exist" containerID="4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925"
Jul 09 19:07:56.413253 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.413209    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925"} err="failed to get container status \"4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925\": rpc error: code = NotFound desc = could not find container \"4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925\": container with ID starting with 4ea228720c480101210d2f4d86fd2486e17450d4f6735434014f27d0eab6f925 not found: ID does not exist"
Jul 09 19:07:56.421323 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.421298    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5b899f7f57-s5p9r"]
Jul 09 19:07:56.425381 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:56.425359    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5b899f7f57-s5p9r"]
Jul 09 19:07:57.877347 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:07:57.877315    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="509d2dae-f425-4cc4-b548-2e56ce22f2fa" path="/var/lib/kubelet/pods/509d2dae-f425-4cc4-b548-2e56ce22f2fa/volumes"
Jul 09 19:08:00.227822 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.227793    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6554b677c6-8p4th"]
Jul 09 19:08:00.228213 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.228142    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="509d2dae-f425-4cc4-b548-2e56ce22f2fa" containerName="authorino"
Jul 09 19:08:00.228213 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.228153    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="509d2dae-f425-4cc4-b548-2e56ce22f2fa" containerName="authorino"
Jul 09 19:08:00.228281 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.228220    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="509d2dae-f425-4cc4-b548-2e56ce22f2fa" containerName="authorino"
Jul 09 19:08:00.234593 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.234572    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.237673 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.237648    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 19:08:00.237673 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.237653    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 19:08:00.237888 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.237701    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rx66n\""
Jul 09 19:08:00.244870 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.244846    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6554b677c6-8p4th"]
Jul 09 19:08:00.313596 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.313561    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmtrv\" (UniqueName: \"kubernetes.io/projected/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-kube-api-access-bmtrv\") pod \"maas-controller-6554b677c6-8p4th\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") " pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.313798 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.313604    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-webhook-cert\") pod \"maas-controller-6554b677c6-8p4th\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") " pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.393093 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.393063    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75c66f9f7-f9qlt"]
Jul 09 19:08:00.396086 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.396065    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.409642 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.409595    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75c66f9f7-f9qlt"]
Jul 09 19:08:00.414940 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.414913    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmtrv\" (UniqueName: \"kubernetes.io/projected/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-kube-api-access-bmtrv\") pod \"maas-controller-6554b677c6-8p4th\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") " pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.415078 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.414952    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-webhook-cert\") pod \"maas-controller-6554b677c6-8p4th\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") " pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.417423 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.417399    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-webhook-cert\") pod \"maas-controller-6554b677c6-8p4th\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") " pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.424001 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.423978    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmtrv\" (UniqueName: \"kubernetes.io/projected/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-kube-api-access-bmtrv\") pod \"maas-controller-6554b677c6-8p4th\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") " pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.516373 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.516281    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-webhook-cert\") pod \"maas-controller-75c66f9f7-f9qlt\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") " pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.516373 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.516318    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t668d\" (UniqueName: \"kubernetes.io/projected/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-kube-api-access-t668d\") pod \"maas-controller-75c66f9f7-f9qlt\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") " pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.546249 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.546218    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:00.617453 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.617421    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-webhook-cert\") pod \"maas-controller-75c66f9f7-f9qlt\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") " pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.617610 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.617468    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t668d\" (UniqueName: \"kubernetes.io/projected/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-kube-api-access-t668d\") pod \"maas-controller-75c66f9f7-f9qlt\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") " pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.619875 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.619851    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-webhook-cert\") pod \"maas-controller-75c66f9f7-f9qlt\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") " pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.627286 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.627263    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t668d\" (UniqueName: \"kubernetes.io/projected/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-kube-api-access-t668d\") pod \"maas-controller-75c66f9f7-f9qlt\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") " pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.682060 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.682038    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6554b677c6-8p4th"]
Jul 09 19:08:00.683965 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:08:00.683937    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09f9cde0_ee02_4aa1_a0bb_6655f98a1e3a.slice/crio-9dcc7d24746f5205fd7b5e9696f0f8bdb2d8f5936567bee153661d2ffa4b238f WatchSource:0}: Error finding container 9dcc7d24746f5205fd7b5e9696f0f8bdb2d8f5936567bee153661d2ffa4b238f: Status 404 returned error can't find the container with id 9dcc7d24746f5205fd7b5e9696f0f8bdb2d8f5936567bee153661d2ffa4b238f
Jul 09 19:08:00.713763 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.713739    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:00.845192 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:00.845161    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75c66f9f7-f9qlt"]
Jul 09 19:08:00.846475 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:08:00.846447    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf278ca0d_1b4a_4d3b_b808_f4f0c4afefca.slice/crio-36291a2c08e8be80f02efdb187ef81ea12b89a687883a31eb7684907d123aa94 WatchSource:0}: Error finding container 36291a2c08e8be80f02efdb187ef81ea12b89a687883a31eb7684907d123aa94: Status 404 returned error can't find the container with id 36291a2c08e8be80f02efdb187ef81ea12b89a687883a31eb7684907d123aa94
Jul 09 19:08:01.437552 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:01.437434    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c66f9f7-f9qlt" event={"ID":"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca","Type":"ContainerStarted","Data":"36291a2c08e8be80f02efdb187ef81ea12b89a687883a31eb7684907d123aa94"}
Jul 09 19:08:01.440146 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:01.440059    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6554b677c6-8p4th" event={"ID":"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a","Type":"ContainerStarted","Data":"9dcc7d24746f5205fd7b5e9696f0f8bdb2d8f5936567bee153661d2ffa4b238f"}
Jul 09 19:08:03.452463 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:03.452430    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6554b677c6-8p4th" event={"ID":"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a","Type":"ContainerStarted","Data":"fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1"}
Jul 09 19:08:03.452989 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:03.452489    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:03.453995 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:03.453971    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c66f9f7-f9qlt" event={"ID":"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca","Type":"ContainerStarted","Data":"089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49"}
Jul 09 19:08:03.454116 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:03.454102    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:03.474518 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:03.474468    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6554b677c6-8p4th" podStartSLOduration=1.161113588 podStartE2EDuration="3.474451391s" totalImagesPullingTime="2.313337803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:08:03.472654689 +0000 UTC m=+526.056086872" watchObservedRunningTime="2026-07-09 19:08:03.474451391 +0000 UTC m=+526.057883571"
Jul 09 19:08:03.501050 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:03.501001    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75c66f9f7-f9qlt" podStartSLOduration=1.344616088 podStartE2EDuration="3.500982851s" totalImagesPullingTime="2.156366763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:08:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:08:03.498332481 +0000 UTC m=+526.081764688" watchObservedRunningTime="2026-07-09 19:08:03.500982851 +0000 UTC m=+526.084415030"
Jul 09 19:08:14.465123 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.465038    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:14.465565 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.465272    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:08:14.529096 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.529067    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6554b677c6-8p4th"]
Jul 09 19:08:14.529340 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.529310    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6554b677c6-8p4th" podUID="09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" containerName="manager" containerID="cri-o://fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1" gracePeriod=30
Jul 09 19:08:14.782043 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.782017    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:14.937791 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.937760    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-webhook-cert\") pod \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") "
Jul 09 19:08:14.937965 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.937842    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmtrv\" (UniqueName: \"kubernetes.io/projected/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-kube-api-access-bmtrv\") pod \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\" (UID: \"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a\") "
Jul 09 19:08:14.940249 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.940211    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" (UID: "09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:08:14.940249 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:14.940232    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-kube-api-access-bmtrv" (OuterVolumeSpecName: "kube-api-access-bmtrv") pod "09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" (UID: "09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a"). InnerVolumeSpecName "kube-api-access-bmtrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:08:15.039077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.039000    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmtrv\" (UniqueName: \"kubernetes.io/projected/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-kube-api-access-bmtrv\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:08:15.039077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.039027    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a-webhook-cert\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:08:15.519667 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.519635    2569 generic.go:356] "Generic (PLEG): container finished" podID="09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" containerID="fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1" exitCode=0
Jul 09 19:08:15.520137 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.519709    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6554b677c6-8p4th" event={"ID":"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a","Type":"ContainerDied","Data":"fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1"}
Jul 09 19:08:15.520137 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.519741    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6554b677c6-8p4th"
Jul 09 19:08:15.520137 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.519752    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6554b677c6-8p4th" event={"ID":"09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a","Type":"ContainerDied","Data":"9dcc7d24746f5205fd7b5e9696f0f8bdb2d8f5936567bee153661d2ffa4b238f"}
Jul 09 19:08:15.520137 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.519776    2569 scope.go:117] "RemoveContainer" containerID="fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1"
Jul 09 19:08:15.529381 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.529362    2569 scope.go:117] "RemoveContainer" containerID="fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1"
Jul 09 19:08:15.529649 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:08:15.529630    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1\": container with ID starting with fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1 not found: ID does not exist" containerID="fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1"
Jul 09 19:08:15.529728 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.529657    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1"} err="failed to get container status \"fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1\": rpc error: code = NotFound desc = could not find container \"fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1\": container with ID starting with fe823ebef8fdfd91b6eb305b652219b3a0ae07577d8a83835e3166ab1f93c7e1 not found: ID does not exist"
Jul 09 19:08:15.543231 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.543208    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6554b677c6-8p4th"]
Jul 09 19:08:15.552202 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.552179    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6554b677c6-8p4th"]
Jul 09 19:08:15.877877 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:15.877847    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" path="/var/lib/kubelet/pods/09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a/volumes"
Jul 09 19:08:38.629895 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.629860    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"]
Jul 09 19:08:38.630397 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.630224    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" containerName="manager"
Jul 09 19:08:38.630397 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.630235    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" containerName="manager"
Jul 09 19:08:38.630397 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.630294    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="09f9cde0-ee02-4aa1-a0bb-6655f98a1e3a" containerName="manager"
Jul 09 19:08:38.657178 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.657145    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"]
Jul 09 19:08:38.657332 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.657243    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.660317 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.660281    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 09 19:08:38.660463 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.660364    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dlwh6\""
Jul 09 19:08:38.738905 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.738871    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-openshift-service-ca\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.739082 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.738914    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-maas-api-tls\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.739082 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.738945    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mq9q\" (UniqueName: \"kubernetes.io/projected/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-kube-api-access-2mq9q\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.840038 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.840009    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-openshift-service-ca\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.840281 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.840047    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-maas-api-tls\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.840281 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.840080    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mq9q\" (UniqueName: \"kubernetes.io/projected/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-kube-api-access-2mq9q\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.840847 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.840820    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-openshift-service-ca\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.843263 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.843234    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-maas-api-tls\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.849184 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.849155    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mq9q\" (UniqueName: \"kubernetes.io/projected/f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f-kube-api-access-2mq9q\") pod \"maas-api-67f4cb5d58-swlsk\" (UID: \"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f\") " pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:38.969086 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:38.968988    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:39.123505 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.123465    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qcf4q"]
Jul 09 19:08:39.129736 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.129709    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.133787 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.133761    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 09 19:08:39.135381 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.135354    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"]
Jul 09 19:08:39.137004 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:08:39.136984    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf25fad68_c8b2_4e37_ac5c_5fa1dc187f6f.slice/crio-5db77a785c28e5bdd37c36da45395bbbfa4b893e0ab15d4c94c7964250dbd56d WatchSource:0}: Error finding container 5db77a785c28e5bdd37c36da45395bbbfa4b893e0ab15d4c94c7964250dbd56d: Status 404 returned error can't find the container with id 5db77a785c28e5bdd37c36da45395bbbfa4b893e0ab15d4c94c7964250dbd56d
Jul 09 19:08:39.138448 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.138425    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-lbzgm"]
Jul 09 19:08:39.142465 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.142444    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.146140 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.146122    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-6rqq5\""
Jul 09 19:08:39.147356 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.147336    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qcf4q"]
Jul 09 19:08:39.171500 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.171479    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-lbzgm"]
Jul 09 19:08:39.243622 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.243544    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d34a207-f5f3-418f-ac31-b4dbdeee2ec2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qcf4q\" (UID: \"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2\") " pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.243622 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.243583    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn4df\" (UniqueName: \"kubernetes.io/projected/9d34a207-f5f3-418f-ac31-b4dbdeee2ec2-kube-api-access-cn4df\") pod \"payload-pre-processing-7964457696-qcf4q\" (UID: \"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2\") " pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.243813 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.243660    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0-plugins-config-volume\") pod \"payload-processing-58f86c7798-lbzgm\" (UID: \"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0\") " pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.243813 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.243708    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tf7v\" (UniqueName: \"kubernetes.io/projected/8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0-kube-api-access-8tf7v\") pod \"payload-processing-58f86c7798-lbzgm\" (UID: \"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0\") " pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.344914 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.344878    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d34a207-f5f3-418f-ac31-b4dbdeee2ec2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qcf4q\" (UID: \"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2\") " pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.344914 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.344917    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn4df\" (UniqueName: \"kubernetes.io/projected/9d34a207-f5f3-418f-ac31-b4dbdeee2ec2-kube-api-access-cn4df\") pod \"payload-pre-processing-7964457696-qcf4q\" (UID: \"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2\") " pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.345145 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.344968    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0-plugins-config-volume\") pod \"payload-processing-58f86c7798-lbzgm\" (UID: \"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0\") " pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.345145 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.344987    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tf7v\" (UniqueName: \"kubernetes.io/projected/8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0-kube-api-access-8tf7v\") pod \"payload-processing-58f86c7798-lbzgm\" (UID: \"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0\") " pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.345631 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.345611    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d34a207-f5f3-418f-ac31-b4dbdeee2ec2-plugins-config-volume\") pod \"payload-pre-processing-7964457696-qcf4q\" (UID: \"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2\") " pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.345740 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.345724    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0-plugins-config-volume\") pod \"payload-processing-58f86c7798-lbzgm\" (UID: \"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0\") " pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.355253 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.355209    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn4df\" (UniqueName: \"kubernetes.io/projected/9d34a207-f5f3-418f-ac31-b4dbdeee2ec2-kube-api-access-cn4df\") pod \"payload-pre-processing-7964457696-qcf4q\" (UID: \"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2\") " pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.355370 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.355336    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tf7v\" (UniqueName: \"kubernetes.io/projected/8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0-kube-api-access-8tf7v\") pod \"payload-processing-58f86c7798-lbzgm\" (UID: \"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0\") " pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.452232 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.452195    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:39.459561 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.459540    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:39.607246 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.607219    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-qcf4q"]
Jul 09 19:08:39.608758 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:08:39.608664    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d34a207_f5f3_418f_ac31_b4dbdeee2ec2.slice/crio-d9aa3b562d2c2a753f991d1c55032a00292e26700d1c0cc481586863429aaa39 WatchSource:0}: Error finding container d9aa3b562d2c2a753f991d1c55032a00292e26700d1c0cc481586863429aaa39: Status 404 returned error can't find the container with id d9aa3b562d2c2a753f991d1c55032a00292e26700d1c0cc481586863429aaa39
Jul 09 19:08:39.627188 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.627164    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-lbzgm"]
Jul 09 19:08:39.628002 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:08:39.627977    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8beec3a6_4d6e_400c_8c1b_95b0b8f7bac0.slice/crio-d1084ccffba3f8fd88f67b547bedfc545d3d00e70c19fce6a0fe36fd2edacba1 WatchSource:0}: Error finding container d1084ccffba3f8fd88f67b547bedfc545d3d00e70c19fce6a0fe36fd2edacba1: Status 404 returned error can't find the container with id d1084ccffba3f8fd88f67b547bedfc545d3d00e70c19fce6a0fe36fd2edacba1
Jul 09 19:08:39.647287 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.647262    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk" event={"ID":"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f","Type":"ContainerStarted","Data":"5db77a785c28e5bdd37c36da45395bbbfa4b893e0ab15d4c94c7964250dbd56d"}
Jul 09 19:08:39.648584 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.648560    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q" event={"ID":"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2","Type":"ContainerStarted","Data":"d9aa3b562d2c2a753f991d1c55032a00292e26700d1c0cc481586863429aaa39"}
Jul 09 19:08:39.649675 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:39.649654    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-lbzgm" event={"ID":"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0","Type":"ContainerStarted","Data":"d1084ccffba3f8fd88f67b547bedfc545d3d00e70c19fce6a0fe36fd2edacba1"}
Jul 09 19:08:41.662539 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.662506    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q" event={"ID":"9d34a207-f5f3-418f-ac31-b4dbdeee2ec2","Type":"ContainerStarted","Data":"2b4d84806d2e7a8a58b7395e4cdf6ec96d0db6053d2737981a341620f474dac7"}
Jul 09 19:08:41.662968 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.662712    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:08:41.664165 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.664140    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-lbzgm" event={"ID":"8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0","Type":"ContainerStarted","Data":"0eff5d52d476ffe6d335e33263dba6cccc09d81640918342138626ca04fd7d93"}
Jul 09 19:08:41.664302 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.664279    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:41.665614 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.665592    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk" event={"ID":"f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f","Type":"ContainerStarted","Data":"8ae52bac93c0b6baf486560985e702a833b2962cfbe0dc65a3df3df423483722"}
Jul 09 19:08:41.665725 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.665631    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:41.696511 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.696402    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q" podStartSLOduration=0.86885337 podStartE2EDuration="2.696385776s" totalImagesPullingTime="1.827532406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:08:41.693107593 +0000 UTC m=+564.276539776" watchObservedRunningTime="2026-07-09 19:08:41.696385776 +0000 UTC m=+564.279817956"
Jul 09 19:08:41.755588 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.755541    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-lbzgm" podStartSLOduration=0.950173803 podStartE2EDuration="2.755525907s" totalImagesPullingTime="1.805352104s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:08:41.728020154 +0000 UTC m=+564.311452334" watchObservedRunningTime="2026-07-09 19:08:41.755525907 +0000 UTC m=+564.338958128"
Jul 09 19:08:41.757061 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:41.757021    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk" podStartSLOduration=1.460049309 podStartE2EDuration="3.757012203s" totalImagesPullingTime="2.296962894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:08:41.755446119 +0000 UTC m=+564.338878299" watchObservedRunningTime="2026-07-09 19:08:41.757012203 +0000 UTC m=+564.340444382"
Jul 09 19:08:47.676612 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:47.676583    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-67f4cb5d58-swlsk"
Jul 09 19:08:52.673272 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:52.673242    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-lbzgm"
Jul 09 19:08:52.673752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:08:52.673556    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-qcf4q"
Jul 09 19:09:01.550951 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.550915    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"]
Jul 09 19:09:01.556040 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.556018    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.559340 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.559320    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 09 19:09:01.559441 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.559420    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 09 19:09:01.560872 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.560828    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-tt5pt\""
Jul 09 19:09:01.561008 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.560900    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 09 19:09:01.564990 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.564972    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"]
Jul 09 19:09:01.738818 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.738786    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.738984 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.738825    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.738984 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.738852    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9qn4c\" (UniqueName: \"kubernetes.io/projected/d3e4344b-9d79-4046-8dea-4698a69b145d-kube-api-access-9qn4c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.738984 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.738922    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.738984 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.738966    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3e4344b-9d79-4046-8dea-4698a69b145d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.739120 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.739003    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.839728 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.839616    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.839728 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.839654    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9qn4c\" (UniqueName: \"kubernetes.io/projected/d3e4344b-9d79-4046-8dea-4698a69b145d-kube-api-access-9qn4c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.839728 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.839676    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.839728 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.839725    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3e4344b-9d79-4046-8dea-4698a69b145d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.840086 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.839846    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.840086 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.839923    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.840211 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.840098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.840211 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.840154    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.840211 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.840165    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.842204 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.842181    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d3e4344b-9d79-4046-8dea-4698a69b145d-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.842479 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.842459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d3e4344b-9d79-4046-8dea-4698a69b145d-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.849055 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.849033    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9qn4c\" (UniqueName: \"kubernetes.io/projected/d3e4344b-9d79-4046-8dea-4698a69b145d-kube-api-access-9qn4c\") pod \"e2e-distinct-simulated-kserve-59975d69f9-rv2x9\" (UID: \"d3e4344b-9d79-4046-8dea-4698a69b145d\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:01.868073 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:01.868046    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:02.014274 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:02.014245    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"]
Jul 09 19:09:02.015759 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:09:02.015735    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e4344b_9d79_4046_8dea_4698a69b145d.slice/crio-d7657db2cf328d5e969fbb29ca9ab65f23874c9e6c758f4f11c7ed6ca575a8b6 WatchSource:0}: Error finding container d7657db2cf328d5e969fbb29ca9ab65f23874c9e6c758f4f11c7ed6ca575a8b6: Status 404 returned error can't find the container with id d7657db2cf328d5e969fbb29ca9ab65f23874c9e6c758f4f11c7ed6ca575a8b6
Jul 09 19:09:02.791003 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:02.790956    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9" event={"ID":"d3e4344b-9d79-4046-8dea-4698a69b145d","Type":"ContainerStarted","Data":"d7657db2cf328d5e969fbb29ca9ab65f23874c9e6c758f4f11c7ed6ca575a8b6"}
Jul 09 19:09:03.798359 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:03.798327    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9" event={"ID":"d3e4344b-9d79-4046-8dea-4698a69b145d","Type":"ContainerStarted","Data":"334cda70189b201a48d7f9e2e6b97177c92c58e0f8151b6785dce5e7cf84127d"}
Jul 09 19:09:03.821751 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:03.821704    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9" podStartSLOduration=1.743077801 podStartE2EDuration="2.821672671s" totalImagesPullingTime="1.07859487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:09:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:09:03.819540037 +0000 UTC m=+586.402972218" watchObservedRunningTime="2026-07-09 19:09:03.821672671 +0000 UTC m=+586.405104851"
Jul 09 19:09:11.868765 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:11.868735    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:11.869306 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:11.869002    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:11.883121 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:11.883094    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:12.857351 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:12.857318    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-rv2x9"
Jul 09 19:09:37.457204 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.457172    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"]
Jul 09 19:09:37.459671 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.459654    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.462404 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.462382    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 09 19:09:37.471203 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.471174    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"]
Jul 09 19:09:37.543775 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.543738    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.543957 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.543842    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.543957 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.543865    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8704231-8e17-4bd6-8782-d5fc80ad613f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.543957 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.543940    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.544102 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.543975    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmd29\" (UniqueName: \"kubernetes.io/projected/e8704231-8e17-4bd6-8782-d5fc80ad613f-kube-api-access-qmd29\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.544102 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.544020    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645186 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645150    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645213    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645231    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8704231-8e17-4bd6-8782-d5fc80ad613f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645257    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645273    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmd29\" (UniqueName: \"kubernetes.io/projected/e8704231-8e17-4bd6-8782-d5fc80ad613f-kube-api-access-qmd29\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645301    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645805 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645663    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645805 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645740    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.645805 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.645760    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.647737 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.647715    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e8704231-8e17-4bd6-8782-d5fc80ad613f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.648041 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.648013    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8704231-8e17-4bd6-8782-d5fc80ad613f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.653249 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.653229    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmd29\" (UniqueName: \"kubernetes.io/projected/e8704231-8e17-4bd6-8782-d5fc80ad613f-kube-api-access-qmd29\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q\" (UID: \"e8704231-8e17-4bd6-8782-d5fc80ad613f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.777977 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.777892    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:37.918125 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.918098    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"]
Jul 09 19:09:37.919716 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:09:37.919671    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8704231_8e17_4bd6_8782_d5fc80ad613f.slice/crio-40994c4cc044a5ab8e643df8aa88f695e27a1820a3badf11568ef4aff2116c81 WatchSource:0}: Error finding container 40994c4cc044a5ab8e643df8aa88f695e27a1820a3badf11568ef4aff2116c81: Status 404 returned error can't find the container with id 40994c4cc044a5ab8e643df8aa88f695e27a1820a3badf11568ef4aff2116c81
Jul 09 19:09:37.920987 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.920970    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 19:09:37.981055 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:37.981021    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q" event={"ID":"e8704231-8e17-4bd6-8782-d5fc80ad613f","Type":"ContainerStarted","Data":"40994c4cc044a5ab8e643df8aa88f695e27a1820a3badf11568ef4aff2116c81"}
Jul 09 19:09:38.988322 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:38.988235    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q" event={"ID":"e8704231-8e17-4bd6-8782-d5fc80ad613f","Type":"ContainerStarted","Data":"2d18ed1f5ee0cf2815fd6208ab5ace529f4c168b01826651e6653ac910d1da8e"}
Jul 09 19:09:39.008225 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:39.008179    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q" podStartSLOduration=1.823320695 podStartE2EDuration="2.008164453s" totalImagesPullingTime="184.843758ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:09:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:09:39.006516323 +0000 UTC m=+621.589948506" watchObservedRunningTime="2026-07-09 19:09:39.008164453 +0000 UTC m=+621.591596670"
Jul 09 19:09:47.778168 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:47.778131    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:47.779002 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:47.778223    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:47.790920 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:47.790898    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:09:48.048052 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:09:48.047968    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q"
Jul 09 19:11:08.910776 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:08.910673    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b44b5c9b-q6n8f"]
Jul 09 19:11:08.913244 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:08.913224    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:08.917276 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:08.917256    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fcl7g\""
Jul 09 19:11:08.917369 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:08.917265    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 09 19:11:08.922507 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:08.922485    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b44b5c9b-q6n8f"]
Jul 09 19:11:08.947813 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:08.947790    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/331e6bc2-9bf9-4f5f-b039-38d480d78378-tls-cert\") pod \"authorino-8b44b5c9b-q6n8f\" (UID: \"331e6bc2-9bf9-4f5f-b039-38d480d78378\") " pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:08.947974 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:08.947850    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcz24\" (UniqueName: \"kubernetes.io/projected/331e6bc2-9bf9-4f5f-b039-38d480d78378-kube-api-access-bcz24\") pod \"authorino-8b44b5c9b-q6n8f\" (UID: \"331e6bc2-9bf9-4f5f-b039-38d480d78378\") " pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:09.049497 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:09.049466    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/331e6bc2-9bf9-4f5f-b039-38d480d78378-tls-cert\") pod \"authorino-8b44b5c9b-q6n8f\" (UID: \"331e6bc2-9bf9-4f5f-b039-38d480d78378\") " pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:09.049712 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:09.049514    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcz24\" (UniqueName: \"kubernetes.io/projected/331e6bc2-9bf9-4f5f-b039-38d480d78378-kube-api-access-bcz24\") pod \"authorino-8b44b5c9b-q6n8f\" (UID: \"331e6bc2-9bf9-4f5f-b039-38d480d78378\") " pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:09.051839 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:09.051818    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/331e6bc2-9bf9-4f5f-b039-38d480d78378-tls-cert\") pod \"authorino-8b44b5c9b-q6n8f\" (UID: \"331e6bc2-9bf9-4f5f-b039-38d480d78378\") " pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:09.058252 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:09.058232    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcz24\" (UniqueName: \"kubernetes.io/projected/331e6bc2-9bf9-4f5f-b039-38d480d78378-kube-api-access-bcz24\") pod \"authorino-8b44b5c9b-q6n8f\" (UID: \"331e6bc2-9bf9-4f5f-b039-38d480d78378\") " pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:09.223893 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:09.223803    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b44b5c9b-q6n8f"
Jul 09 19:11:09.372194 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:09.372164    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b44b5c9b-q6n8f"]
Jul 09 19:11:09.373328 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:11:09.373297    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod331e6bc2_9bf9_4f5f_b039_38d480d78378.slice/crio-795f0c8cd0e1ab91b98b2b27f49361d48412b86472945419bb78ba3d11c3b56e WatchSource:0}: Error finding container 795f0c8cd0e1ab91b98b2b27f49361d48412b86472945419bb78ba3d11c3b56e: Status 404 returned error can't find the container with id 795f0c8cd0e1ab91b98b2b27f49361d48412b86472945419bb78ba3d11c3b56e
Jul 09 19:11:09.495587 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:09.495504    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b44b5c9b-q6n8f" event={"ID":"331e6bc2-9bf9-4f5f-b039-38d480d78378","Type":"ContainerStarted","Data":"795f0c8cd0e1ab91b98b2b27f49361d48412b86472945419bb78ba3d11c3b56e"}
Jul 09 19:11:10.504761 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:10.504719    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b44b5c9b-q6n8f" event={"ID":"331e6bc2-9bf9-4f5f-b039-38d480d78378","Type":"ContainerStarted","Data":"36960f8a52df0ab3df1e072be7870bf6289a4d36acefdcc500b631116aaeacde"}
Jul 09 19:11:10.528553 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:11:10.528493    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8b44b5c9b-q6n8f" podStartSLOduration=1.949891078 podStartE2EDuration="2.528470534s" totalImagesPullingTime="578.579456ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:11:10.525378987 +0000 UTC m=+713.108811189" watchObservedRunningTime="2026-07-09 19:11:10.528470534 +0000 UTC m=+713.111902716"
Jul 09 19:12:20.186614 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.186565    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75c66f9f7-f9qlt"]
Jul 09 19:12:20.187187 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.186845    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-75c66f9f7-f9qlt" podUID="f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" containerName="manager" containerID="cri-o://089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49" gracePeriod=30
Jul 09 19:12:20.444980 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.444924    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:12:20.572484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.572443    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t668d\" (UniqueName: \"kubernetes.io/projected/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-kube-api-access-t668d\") pod \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") "
Jul 09 19:12:20.572660 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.572524    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-webhook-cert\") pod \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\" (UID: \"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca\") "
Jul 09 19:12:20.574713 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.574656    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" (UID: "f278ca0d-1b4a-4d3b-b808-f4f0c4afefca"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:12:20.574713 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.574701    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-kube-api-access-t668d" (OuterVolumeSpecName: "kube-api-access-t668d") pod "f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" (UID: "f278ca0d-1b4a-4d3b-b808-f4f0c4afefca"). InnerVolumeSpecName "kube-api-access-t668d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:12:20.673664 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.673632    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t668d\" (UniqueName: \"kubernetes.io/projected/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-kube-api-access-t668d\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:12:20.673664 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.673659    2569 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca-webhook-cert\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:12:20.909235 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.909193    2569 generic.go:356] "Generic (PLEG): container finished" podID="f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" containerID="089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49" exitCode=0
Jul 09 19:12:20.909403 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.909253    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c66f9f7-f9qlt"
Jul 09 19:12:20.909403 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.909276    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c66f9f7-f9qlt" event={"ID":"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca","Type":"ContainerDied","Data":"089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49"}
Jul 09 19:12:20.909403 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.909349    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c66f9f7-f9qlt" event={"ID":"f278ca0d-1b4a-4d3b-b808-f4f0c4afefca","Type":"ContainerDied","Data":"36291a2c08e8be80f02efdb187ef81ea12b89a687883a31eb7684907d123aa94"}
Jul 09 19:12:20.909403 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.909372    2569 scope.go:117] "RemoveContainer" containerID="089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49"
Jul 09 19:12:20.919625 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.919604    2569 scope.go:117] "RemoveContainer" containerID="089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49"
Jul 09 19:12:20.920027 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:12:20.919996    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49\": container with ID starting with 089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49 not found: ID does not exist" containerID="089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49"
Jul 09 19:12:20.920117 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.920050    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49"} err="failed to get container status \"089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49\": rpc error: code = NotFound desc = could not find container \"089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49\": container with ID starting with 089a9bbeec7fce75293d1fc6c8c4ff3c83d0ee256ba27d0263dd42b7ce2fef49 not found: ID does not exist"
Jul 09 19:12:20.932735 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.932715    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-75c66f9f7-f9qlt"]
Jul 09 19:12:20.937040 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:20.937020    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-75c66f9f7-f9qlt"]
Jul 09 19:12:21.685253 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.685220    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-75c66f9f7-sz7zm"]
Jul 09 19:12:21.685626 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.685615    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" containerName="manager"
Jul 09 19:12:21.685675 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.685628    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" containerName="manager"
Jul 09 19:12:21.685739 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.685713    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" containerName="manager"
Jul 09 19:12:21.688718 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.688701    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:21.693237 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.693214    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 09 19:12:21.693379 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.693246    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 09 19:12:21.693379 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.693214    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-rx66n\""
Jul 09 19:12:21.699984 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.699963    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75c66f9f7-sz7zm"]
Jul 09 19:12:21.782704 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.782660    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/435bd819-2d30-4e95-af9d-842eef51b157-webhook-cert\") pod \"maas-controller-75c66f9f7-sz7zm\" (UID: \"435bd819-2d30-4e95-af9d-842eef51b157\") " pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:21.782843 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.782726    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd6q9\" (UniqueName: \"kubernetes.io/projected/435bd819-2d30-4e95-af9d-842eef51b157-kube-api-access-wd6q9\") pod \"maas-controller-75c66f9f7-sz7zm\" (UID: \"435bd819-2d30-4e95-af9d-842eef51b157\") " pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:21.879554 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.879524    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f278ca0d-1b4a-4d3b-b808-f4f0c4afefca" path="/var/lib/kubelet/pods/f278ca0d-1b4a-4d3b-b808-f4f0c4afefca/volumes"
Jul 09 19:12:21.883375 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.883347    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/435bd819-2d30-4e95-af9d-842eef51b157-webhook-cert\") pod \"maas-controller-75c66f9f7-sz7zm\" (UID: \"435bd819-2d30-4e95-af9d-842eef51b157\") " pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:21.883539 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.883405    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd6q9\" (UniqueName: \"kubernetes.io/projected/435bd819-2d30-4e95-af9d-842eef51b157-kube-api-access-wd6q9\") pod \"maas-controller-75c66f9f7-sz7zm\" (UID: \"435bd819-2d30-4e95-af9d-842eef51b157\") " pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:21.886061 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.886031    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/435bd819-2d30-4e95-af9d-842eef51b157-webhook-cert\") pod \"maas-controller-75c66f9f7-sz7zm\" (UID: \"435bd819-2d30-4e95-af9d-842eef51b157\") " pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:21.893090 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:21.893070    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd6q9\" (UniqueName: \"kubernetes.io/projected/435bd819-2d30-4e95-af9d-842eef51b157-kube-api-access-wd6q9\") pod \"maas-controller-75c66f9f7-sz7zm\" (UID: \"435bd819-2d30-4e95-af9d-842eef51b157\") " pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:22.001881 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:22.001802    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:22.139741 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:22.139708    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-75c66f9f7-sz7zm"]
Jul 09 19:12:22.141231 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:12:22.141206    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435bd819_2d30_4e95_af9d_842eef51b157.slice/crio-59d07e5ad347e24d92a06b041b0f30e8a050971b1977d15ed51ed0ed6c5cd69b WatchSource:0}: Error finding container 59d07e5ad347e24d92a06b041b0f30e8a050971b1977d15ed51ed0ed6c5cd69b: Status 404 returned error can't find the container with id 59d07e5ad347e24d92a06b041b0f30e8a050971b1977d15ed51ed0ed6c5cd69b
Jul 09 19:12:22.922217 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:22.922181    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c66f9f7-sz7zm" event={"ID":"435bd819-2d30-4e95-af9d-842eef51b157","Type":"ContainerStarted","Data":"14bb128ad9f8be8a97a296de3b99a6e3625a61a36bc034699947908932de05af"}
Jul 09 19:12:22.922563 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:22.922245    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-75c66f9f7-sz7zm" event={"ID":"435bd819-2d30-4e95-af9d-842eef51b157","Type":"ContainerStarted","Data":"59d07e5ad347e24d92a06b041b0f30e8a050971b1977d15ed51ed0ed6c5cd69b"}
Jul 09 19:12:22.922563 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:22.922267    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:12:22.945821 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:22.945780    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-75c66f9f7-sz7zm" podStartSLOduration=1.352964731 podStartE2EDuration="1.945765051s" totalImagesPullingTime="592.80032ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:12:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:12:22.94461909 +0000 UTC m=+785.528051271" watchObservedRunningTime="2026-07-09 19:12:22.945765051 +0000 UTC m=+785.529197231"
Jul 09 19:12:33.933509 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:12:33.933469    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-75c66f9f7-sz7zm"
Jul 09 19:17:13.879439 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:13.879346    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"]
Jul 09 19:17:13.880028 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:13.879598    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94" podUID="141e8fc2-c7a1-4574-a413-772ab1554519" containerName="manager" containerID="cri-o://0ca0b8980eb163c2b9777093f10d9fd4a1e199629c8e1090f8f3565a3ebf1ad3" gracePeriod=10
Jul 09 19:17:17.557492 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.557463    2569 generic.go:356] "Generic (PLEG): container finished" podID="141e8fc2-c7a1-4574-a413-772ab1554519" containerID="0ca0b8980eb163c2b9777093f10d9fd4a1e199629c8e1090f8f3565a3ebf1ad3" exitCode=0
Jul 09 19:17:17.557859 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.557521    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94" event={"ID":"141e8fc2-c7a1-4574-a413-772ab1554519","Type":"ContainerDied","Data":"0ca0b8980eb163c2b9777093f10d9fd4a1e199629c8e1090f8f3565a3ebf1ad3"}
Jul 09 19:17:17.647371 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.647347    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:17:17.706746 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.706715    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/141e8fc2-c7a1-4574-a413-772ab1554519-extensions-socket-volume\") pod \"141e8fc2-c7a1-4574-a413-772ab1554519\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") "
Jul 09 19:17:17.706919 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.706778    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dt4w\" (UniqueName: \"kubernetes.io/projected/141e8fc2-c7a1-4574-a413-772ab1554519-kube-api-access-9dt4w\") pod \"141e8fc2-c7a1-4574-a413-772ab1554519\" (UID: \"141e8fc2-c7a1-4574-a413-772ab1554519\") "
Jul 09 19:17:17.707153 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.707127    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/141e8fc2-c7a1-4574-a413-772ab1554519-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "141e8fc2-c7a1-4574-a413-772ab1554519" (UID: "141e8fc2-c7a1-4574-a413-772ab1554519"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:17:17.709074 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.709043    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/141e8fc2-c7a1-4574-a413-772ab1554519-kube-api-access-9dt4w" (OuterVolumeSpecName: "kube-api-access-9dt4w") pod "141e8fc2-c7a1-4574-a413-772ab1554519" (UID: "141e8fc2-c7a1-4574-a413-772ab1554519"). InnerVolumeSpecName "kube-api-access-9dt4w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:17:17.807827 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.807734    2569 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/141e8fc2-c7a1-4574-a413-772ab1554519-extensions-socket-volume\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:17:17.807827 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.807772    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dt4w\" (UniqueName: \"kubernetes.io/projected/141e8fc2-c7a1-4574-a413-772ab1554519-kube-api-access-9dt4w\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:17:17.952903 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:17.952874    2569 scope.go:117] "RemoveContainer" containerID="0ca0b8980eb163c2b9777093f10d9fd4a1e199629c8e1090f8f3565a3ebf1ad3"
Jul 09 19:17:18.562611 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:18.562578    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"
Jul 09 19:17:18.562611 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:18.562605    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94" event={"ID":"141e8fc2-c7a1-4574-a413-772ab1554519","Type":"ContainerDied","Data":"3604b086e53b220c37f57b0042972ad294e3a65675f68e5d7ab33c018c7e1de6"}
Jul 09 19:17:18.596075 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:18.596048    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"]
Jul 09 19:17:18.600753 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:18.600729    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-5rw94"]
Jul 09 19:17:19.877426 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:17:19.877390    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="141e8fc2-c7a1-4574-a413-772ab1554519" path="/var/lib/kubelet/pods/141e8fc2-c7a1-4574-a413-772ab1554519/volumes"
Jul 09 19:18:15.012186 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.012149    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"]
Jul 09 19:18:15.012656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.012548    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="141e8fc2-c7a1-4574-a413-772ab1554519" containerName="manager"
Jul 09 19:18:15.012656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.012559    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="141e8fc2-c7a1-4574-a413-772ab1554519" containerName="manager"
Jul 09 19:18:15.012656 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.012625    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="141e8fc2-c7a1-4574-a413-772ab1554519" containerName="manager"
Jul 09 19:18:15.015714 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.015698    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.018971 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.018949    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mh8f8\""
Jul 09 19:18:15.027154 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.027132    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"]
Jul 09 19:18:15.071294 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.071269    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn8nv\" (UniqueName: \"kubernetes.io/projected/98f45300-a79c-4b63-bc9c-cfdf2a2c2b92-kube-api-access-jn8nv\") pod \"kuadrant-operator-controller-manager-86878c769f-bnqcn\" (UID: \"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.071448 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.071389    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/98f45300-a79c-4b63-bc9c-cfdf2a2c2b92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bnqcn\" (UID: \"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.172570 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.172543    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/98f45300-a79c-4b63-bc9c-cfdf2a2c2b92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bnqcn\" (UID: \"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.172726 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.172646    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn8nv\" (UniqueName: \"kubernetes.io/projected/98f45300-a79c-4b63-bc9c-cfdf2a2c2b92-kube-api-access-jn8nv\") pod \"kuadrant-operator-controller-manager-86878c769f-bnqcn\" (UID: \"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.172957 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.172939    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/98f45300-a79c-4b63-bc9c-cfdf2a2c2b92-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-bnqcn\" (UID: \"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.185627 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.185596    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn8nv\" (UniqueName: \"kubernetes.io/projected/98f45300-a79c-4b63-bc9c-cfdf2a2c2b92-kube-api-access-jn8nv\") pod \"kuadrant-operator-controller-manager-86878c769f-bnqcn\" (UID: \"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.326882 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.326805    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.480841 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.480811    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"]
Jul 09 19:18:15.481477 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:18:15.481456    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98f45300_a79c_4b63_bc9c_cfdf2a2c2b92.slice/crio-78bf131c1511cd9fceab44a41e8705603b92cd1bcebe25938f87f971a74d4c64 WatchSource:0}: Error finding container 78bf131c1511cd9fceab44a41e8705603b92cd1bcebe25938f87f971a74d4c64: Status 404 returned error can't find the container with id 78bf131c1511cd9fceab44a41e8705603b92cd1bcebe25938f87f971a74d4c64
Jul 09 19:18:15.885923 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.885875    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn" event={"ID":"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92","Type":"ContainerStarted","Data":"bf050f3db1ca2e44490253a5bba65010eb17cec59589aba3528ba3a30095635f"}
Jul 09 19:18:15.886138 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.885954    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn" event={"ID":"98f45300-a79c-4b63-bc9c-cfdf2a2c2b92","Type":"ContainerStarted","Data":"78bf131c1511cd9fceab44a41e8705603b92cd1bcebe25938f87f971a74d4c64"}
Jul 09 19:18:15.886206 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.886143    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:18:15.909109 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:15.909061    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn" podStartSLOduration=1.909046435 podStartE2EDuration="1.909046435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:18:15.907802812 +0000 UTC m=+1138.491234991" watchObservedRunningTime="2026-07-09 19:18:15.909046435 +0000 UTC m=+1138.492478609"
Jul 09 19:18:26.893832 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:18:26.893799    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-bnqcn"
Jul 09 19:31:49.645871 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.645839    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"]
Jul 09 19:31:49.649245 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.649223    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.653800 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.653780    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4038d9\"/\"kube-root-ca.crt\""
Jul 09 19:31:49.653905 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.653835    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-4038d9\"/\"openshift-service-ca.crt\""
Jul 09 19:31:49.653965 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.653909    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4038d9\"/\"test-model-4038d9-kserve-self-signed-certs\""
Jul 09 19:31:49.653965 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.653912    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-4038d9\"/\"default-dockercfg-lchdp\""
Jul 09 19:31:49.660259 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.660237    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"]
Jul 09 19:31:49.819063 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.819036    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-model-cache\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.819243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.819070    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tls-certs\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.819243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.819094    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-dshm\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.819243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.819114    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztdhq\" (UniqueName: \"kubernetes.io/projected/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-kube-api-access-ztdhq\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.819243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.819202    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-home\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.819243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.819230    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tmp-dir\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920437 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920355    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-home\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920437 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920393    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tmp-dir\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920437 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920432    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-model-cache\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920675 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920458    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tls-certs\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920675 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920492    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-dshm\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920675 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920521    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztdhq\" (UniqueName: \"kubernetes.io/projected/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-kube-api-access-ztdhq\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920911 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920848    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-home\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.920911 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920862    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tmp-dir\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.921018 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.920925    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-model-cache\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.922950 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.922925    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-dshm\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.923114 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.923098    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tls-certs\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.929538 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.929511    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztdhq\" (UniqueName: \"kubernetes.io/projected/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-kube-api-access-ztdhq\") pod \"test-model-4038d9-kserve-6856f7d6d5-ppqgs\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") " pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:49.960449 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:49.960425    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:50.101907 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:50.101883    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"]
Jul 09 19:31:50.102248 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:31:50.102224    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bf1c2cc_0fb9_4ea6_b8c3_6ddc7e1f0aa7.slice/crio-91871fcf211f628328461149486ede4623cb94d4b2ad7b4a2f3fe6d46852aa8c WatchSource:0}: Error finding container 91871fcf211f628328461149486ede4623cb94d4b2ad7b4a2f3fe6d46852aa8c: Status 404 returned error can't find the container with id 91871fcf211f628328461149486ede4623cb94d4b2ad7b4a2f3fe6d46852aa8c
Jul 09 19:31:50.470165 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:50.470124    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs" event={"ID":"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7","Type":"ContainerStarted","Data":"370998393fbb10cf6e035957a9e1e19884eafe8bf1a5f06e5083097f46f90194"}
Jul 09 19:31:50.470348 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:50.470180    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs" event={"ID":"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7","Type":"ContainerStarted","Data":"91871fcf211f628328461149486ede4623cb94d4b2ad7b4a2f3fe6d46852aa8c"}
Jul 09 19:31:50.489025 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:50.488931    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs" podStartSLOduration=1.488913628 podStartE2EDuration="1.488913628s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:31:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:31:50.487907804 +0000 UTC m=+1953.071339986" watchObservedRunningTime="2026-07-09 19:31:50.488913628 +0000 UTC m=+1953.072345810"
Jul 09 19:31:59.961372 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:59.961327    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:59.961836 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:59.961384    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:31:59.973975 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:31:59.973955    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:32:00.537147 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:00.537123    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:32:08.473117 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:08.472922    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"]
Jul 09 19:32:23.499532 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.499464    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs" podUID="4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" containerName="main" containerID="cri-o://370998393fbb10cf6e035957a9e1e19884eafe8bf1a5f06e5083097f46f90194" gracePeriod=45
Jul 09 19:32:23.657506 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.657467    2569 generic.go:356] "Generic (PLEG): container finished" podID="4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" containerID="370998393fbb10cf6e035957a9e1e19884eafe8bf1a5f06e5083097f46f90194" exitCode=0
Jul 09 19:32:23.657666 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.657506    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs" event={"ID":"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7","Type":"ContainerDied","Data":"370998393fbb10cf6e035957a9e1e19884eafe8bf1a5f06e5083097f46f90194"}
Jul 09 19:32:23.763837 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.763780    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:32:23.814936 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.814905    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tls-certs\") pod \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") "
Jul 09 19:32:23.815077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.814941    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-dshm\") pod \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") "
Jul 09 19:32:23.815077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.814984    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-home\") pod \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") "
Jul 09 19:32:23.815077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.815005    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-model-cache\") pod \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") "
Jul 09 19:32:23.815077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.815036    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tmp-dir\") pod \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") "
Jul 09 19:32:23.815077 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.815069    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ztdhq\" (UniqueName: \"kubernetes.io/projected/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-kube-api-access-ztdhq\") pod \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\" (UID: \"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7\") "
Jul 09 19:32:23.815341 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.815252    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-home" (OuterVolumeSpecName: "home") pod "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" (UID: "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:32:23.815341 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.815324    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-model-cache" (OuterVolumeSpecName: "model-cache") pod "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" (UID: "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:32:23.815434 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.815347    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" (UID: "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:32:23.817318 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.817297    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" (UID: "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:32:23.817445 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.817377    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-kube-api-access-ztdhq" (OuterVolumeSpecName: "kube-api-access-ztdhq") pod "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" (UID: "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7"). InnerVolumeSpecName "kube-api-access-ztdhq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:32:23.817445 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.817433    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-dshm" (OuterVolumeSpecName: "dshm") pod "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" (UID: "4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:32:23.916466 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.916437    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ztdhq\" (UniqueName: \"kubernetes.io/projected/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-kube-api-access-ztdhq\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:32:23.916466 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.916462    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tls-certs\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:32:23.916628 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.916471    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-dshm\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:32:23.916628 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.916479    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-home\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:32:23.916628 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.916487    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-model-cache\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:32:23.916628 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:23.916494    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7-tmp-dir\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:32:24.664050 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:24.664023    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"
Jul 09 19:32:24.664437 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:24.664061    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs" event={"ID":"4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7","Type":"ContainerDied","Data":"91871fcf211f628328461149486ede4623cb94d4b2ad7b4a2f3fe6d46852aa8c"}
Jul 09 19:32:24.664437 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:24.664112    2569 scope.go:117] "RemoveContainer" containerID="370998393fbb10cf6e035957a9e1e19884eafe8bf1a5f06e5083097f46f90194"
Jul 09 19:32:24.685370 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:24.685341    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"]
Jul 09 19:32:24.690521 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:24.690497    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-4038d9/test-model-4038d9-kserve-6856f7d6d5-ppqgs"]
Jul 09 19:32:25.878383 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:32:25.878350    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" path="/var/lib/kubelet/pods/4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7/volumes"
Jul 09 19:34:40.469658 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.469628    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"]
Jul 09 19:34:40.470142 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.470053    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" containerName="main"
Jul 09 19:34:40.470142 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.470066    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" containerName="main"
Jul 09 19:34:40.470217 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.470148    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bf1c2cc-0fb9-4ea6-b8c3-6ddc7e1f0aa7" containerName="main"
Jul 09 19:34:40.473143 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.473126    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.477202 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.477179    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"default-dockercfg-pxj7q\""
Jul 09 19:34:40.478729 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.478710    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"openshift-service-ca.crt\""
Jul 09 19:34:40.478834 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.478740    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"kube-root-ca.crt\""
Jul 09 19:34:40.478834 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.478770    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"auth-test-model-a94244-kserve-self-signed-certs\""
Jul 09 19:34:40.489263 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.489243    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"]
Jul 09 19:34:40.495140 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.495120    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-home\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.495247 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.495146    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tmp-dir\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.495247 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.495165    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95tvx\" (UniqueName: \"kubernetes.io/projected/7b95cbf6-2d75-4f35-a1a6-534cc0735146-kube-api-access-95tvx\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.495247 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.495191    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-dshm\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.495415 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.495298    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-model-cache\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.495415 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.495347    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tls-certs\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.600752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.600712    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-model-cache\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.600752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.600755    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tls-certs\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.600999 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.600808    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-home\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.600999 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.600825    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tmp-dir\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.600999 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.600957    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95tvx\" (UniqueName: \"kubernetes.io/projected/7b95cbf6-2d75-4f35-a1a6-534cc0735146-kube-api-access-95tvx\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.601157 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.601026    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-dshm\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.601157 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.601147    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-model-cache\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.601253 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.601159    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-home\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.601253 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.601226    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tmp-dir\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.603200 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.603176    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-dshm\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.603481 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.603459    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tls-certs\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.610996 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.610962    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95tvx\" (UniqueName: \"kubernetes.io/projected/7b95cbf6-2d75-4f35-a1a6-534cc0735146-kube-api-access-95tvx\") pod \"auth-test-model-a94244-kserve-67c5b6bcb7-64cnk\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") " pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.782309 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.782225    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:40.922179 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:40.922151    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"]
Jul 09 19:34:40.923106 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:34:40.923082    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7b95cbf6_2d75_4f35_a1a6_534cc0735146.slice/crio-955ae17c460a9cb8f376bce2b532863c17bc53f495337cf2b8656b3e9913413f WatchSource:0}: Error finding container 955ae17c460a9cb8f376bce2b532863c17bc53f495337cf2b8656b3e9913413f: Status 404 returned error can't find the container with id 955ae17c460a9cb8f376bce2b532863c17bc53f495337cf2b8656b3e9913413f
Jul 09 19:34:41.421449 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:41.421413    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk" event={"ID":"7b95cbf6-2d75-4f35-a1a6-534cc0735146","Type":"ContainerStarted","Data":"7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c"}
Jul 09 19:34:41.421658 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:41.421464    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk" event={"ID":"7b95cbf6-2d75-4f35-a1a6-534cc0735146","Type":"ContainerStarted","Data":"955ae17c460a9cb8f376bce2b532863c17bc53f495337cf2b8656b3e9913413f"}
Jul 09 19:34:41.441775 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:41.441725    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk" podStartSLOduration=1.441708637 podStartE2EDuration="1.441708637s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:34:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:34:41.439600591 +0000 UTC m=+2124.023032775" watchObservedRunningTime="2026-07-09 19:34:41.441708637 +0000 UTC m=+2124.025140817"
Jul 09 19:34:50.783289 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:50.783257    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:50.783289 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:50.783297    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:50.801450 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:50.801426    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:34:51.487877 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:34:51.487838    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:37:07.953321 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:07.953288    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"]
Jul 09 19:37:22.974481 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:22.974433    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk" podUID="7b95cbf6-2d75-4f35-a1a6-534cc0735146" containerName="main" containerID="cri-o://7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c" gracePeriod=45
Jul 09 19:37:23.242392 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.242364    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:37:23.325262 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.325225    2569 generic.go:356] "Generic (PLEG): container finished" podID="7b95cbf6-2d75-4f35-a1a6-534cc0735146" containerID="7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c" exitCode=0
Jul 09 19:37:23.325438 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.325268    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"
Jul 09 19:37:23.325438 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.325317    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk" event={"ID":"7b95cbf6-2d75-4f35-a1a6-534cc0735146","Type":"ContainerDied","Data":"7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c"}
Jul 09 19:37:23.325438 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.325365    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk" event={"ID":"7b95cbf6-2d75-4f35-a1a6-534cc0735146","Type":"ContainerDied","Data":"955ae17c460a9cb8f376bce2b532863c17bc53f495337cf2b8656b3e9913413f"}
Jul 09 19:37:23.325438 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.325384    2569 scope.go:117] "RemoveContainer" containerID="7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c"
Jul 09 19:37:23.334567 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.334547    2569 scope.go:117] "RemoveContainer" containerID="7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c"
Jul 09 19:37:23.334816 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:37:23.334797    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c\": container with ID starting with 7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c not found: ID does not exist" containerID="7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c"
Jul 09 19:37:23.334870 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.334824    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c"} err="failed to get container status \"7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c\": rpc error: code = NotFound desc = could not find container \"7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c\": container with ID starting with 7eece00e3f37603a2044460d5092c197e5583e5199da501540293ee1a473540c not found: ID does not exist"
Jul 09 19:37:23.435450 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435424    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tmp-dir\") pod \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") "
Jul 09 19:37:23.435620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435482    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-home\") pod \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") "
Jul 09 19:37:23.435620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435513    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-dshm\") pod \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") "
Jul 09 19:37:23.435620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435551    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-95tvx\" (UniqueName: \"kubernetes.io/projected/7b95cbf6-2d75-4f35-a1a6-534cc0735146-kube-api-access-95tvx\") pod \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") "
Jul 09 19:37:23.435620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435568    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-model-cache\") pod \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") "
Jul 09 19:37:23.435620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435590    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tls-certs\") pod \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\" (UID: \"7b95cbf6-2d75-4f35-a1a6-534cc0735146\") "
Jul 09 19:37:23.435983 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435759    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7b95cbf6-2d75-4f35-a1a6-534cc0735146" (UID: "7b95cbf6-2d75-4f35-a1a6-534cc0735146"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:37:23.435983 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435854    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-home" (OuterVolumeSpecName: "home") pod "7b95cbf6-2d75-4f35-a1a6-534cc0735146" (UID: "7b95cbf6-2d75-4f35-a1a6-534cc0735146"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:37:23.435983 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.435863    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-model-cache" (OuterVolumeSpecName: "model-cache") pod "7b95cbf6-2d75-4f35-a1a6-534cc0735146" (UID: "7b95cbf6-2d75-4f35-a1a6-534cc0735146"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:37:23.437904 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.437879    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b95cbf6-2d75-4f35-a1a6-534cc0735146-kube-api-access-95tvx" (OuterVolumeSpecName: "kube-api-access-95tvx") pod "7b95cbf6-2d75-4f35-a1a6-534cc0735146" (UID: "7b95cbf6-2d75-4f35-a1a6-534cc0735146"). InnerVolumeSpecName "kube-api-access-95tvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:37:23.437992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.437891    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7b95cbf6-2d75-4f35-a1a6-534cc0735146" (UID: "7b95cbf6-2d75-4f35-a1a6-534cc0735146"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:37:23.437992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.437900    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-dshm" (OuterVolumeSpecName: "dshm") pod "7b95cbf6-2d75-4f35-a1a6-534cc0735146" (UID: "7b95cbf6-2d75-4f35-a1a6-534cc0735146"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:37:23.536937 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.536856    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tmp-dir\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:37:23.536937 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.536885    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-home\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:37:23.536937 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.536894    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-dshm\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:37:23.536937 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.536902    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-95tvx\" (UniqueName: \"kubernetes.io/projected/7b95cbf6-2d75-4f35-a1a6-534cc0735146-kube-api-access-95tvx\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:37:23.536937 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.536912    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7b95cbf6-2d75-4f35-a1a6-534cc0735146-model-cache\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:37:23.536937 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.536920    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7b95cbf6-2d75-4f35-a1a6-534cc0735146-tls-certs\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:37:23.648880 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.648845    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"]
Jul 09 19:37:23.652831 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.652805    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/auth-test-model-a94244-kserve-67c5b6bcb7-64cnk"]
Jul 09 19:37:23.878523 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:23.878490    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b95cbf6-2d75-4f35-a1a6-534cc0735146" path="/var/lib/kubelet/pods/7b95cbf6-2d75-4f35-a1a6-534cc0735146/volumes"
Jul 09 19:37:30.237170 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.237136    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"]
Jul 09 19:37:30.237758 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.237739    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b95cbf6-2d75-4f35-a1a6-534cc0735146" containerName="main"
Jul 09 19:37:30.237758 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.237759    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b95cbf6-2d75-4f35-a1a6-534cc0735146" containerName="main"
Jul 09 19:37:30.237902 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.237878    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b95cbf6-2d75-4f35-a1a6-534cc0735146" containerName="main"
Jul 09 19:37:30.243596 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.243568    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.248786 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.248748    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"sub-test-model-a94244-kserve-self-signed-certs\""
Jul 09 19:37:30.249036 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.249014    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"openshift-service-ca.crt\""
Jul 09 19:37:30.249242 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.249218    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"default-dockercfg-pxj7q\""
Jul 09 19:37:30.249358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.249313    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-a94244\"/\"kube-root-ca.crt\""
Jul 09 19:37:30.252774 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.252749    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"]
Jul 09 19:37:30.291992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.291959    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18d48980-2e73-4bad-af49-c40d46402c3f-tls-certs\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.291992 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.291992    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfs7s\" (UniqueName: \"kubernetes.io/projected/18d48980-2e73-4bad-af49-c40d46402c3f-kube-api-access-qfs7s\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.292180 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.292024    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-home\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.292180 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.292132    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-dshm\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.292180 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.292168    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-tmp-dir\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.292309 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.292190    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-model-cache\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393198 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393157    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18d48980-2e73-4bad-af49-c40d46402c3f-tls-certs\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393198 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393190    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfs7s\" (UniqueName: \"kubernetes.io/projected/18d48980-2e73-4bad-af49-c40d46402c3f-kube-api-access-qfs7s\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393453 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393218    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-home\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393453 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393258    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-dshm\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393453 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393279    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-tmp-dir\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393453 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393298    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-model-cache\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393759 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393733    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-home\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393828 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393790    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-model-cache\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.393828 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.393808    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-tmp-dir\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.395615 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.395593    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-dshm\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.395967 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.395947    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18d48980-2e73-4bad-af49-c40d46402c3f-tls-certs\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.401472 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.401455    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfs7s\" (UniqueName: \"kubernetes.io/projected/18d48980-2e73-4bad-af49-c40d46402c3f-kube-api-access-qfs7s\") pod \"sub-test-model-a94244-kserve-5975ddc9d7-z9jk2\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") " pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.556631 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.556539    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:30.696254 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:30.696224    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"]
Jul 09 19:37:30.697278 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:37:30.697249    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18d48980_2e73_4bad_af49_c40d46402c3f.slice/crio-932b4165f21fa273fe157c7decb1180cac546dd406b907f8e6ca6ed1e26e3531 WatchSource:0}: Error finding container 932b4165f21fa273fe157c7decb1180cac546dd406b907f8e6ca6ed1e26e3531: Status 404 returned error can't find the container with id 932b4165f21fa273fe157c7decb1180cac546dd406b907f8e6ca6ed1e26e3531
Jul 09 19:37:31.372153 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:31.372118    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2" event={"ID":"18d48980-2e73-4bad-af49-c40d46402c3f","Type":"ContainerStarted","Data":"30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4"}
Jul 09 19:37:31.372524 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:31.372163    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2" event={"ID":"18d48980-2e73-4bad-af49-c40d46402c3f","Type":"ContainerStarted","Data":"932b4165f21fa273fe157c7decb1180cac546dd406b907f8e6ca6ed1e26e3531"}
Jul 09 19:37:31.392035 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:31.391989    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2" podStartSLOduration=1.391972921 podStartE2EDuration="1.391972921s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:37:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:37:31.389557475 +0000 UTC m=+2293.972989655" watchObservedRunningTime="2026-07-09 19:37:31.391972921 +0000 UTC m=+2293.975405101"
Jul 09 19:37:40.557282 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:40.557245    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:40.557739 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:40.557322    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:40.569823 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:40.569798    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:37:41.446533 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:37:41.446506    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:38:16.731207 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:16.731112    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"]
Jul 09 19:38:30.660299 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.660259    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"]
Jul 09 19:38:30.664654 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.664634    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.667854 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.667828    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c8412b\"/\"kube-root-ca.crt\""
Jul 09 19:38:30.669634 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.669553    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c8412b\"/\"openshift-service-ca.crt\""
Jul 09 19:38:30.669634 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.669569    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c8412b\"/\"default-dockercfg-2j4nj\""
Jul 09 19:38:30.669634 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.669623    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c8412b\"/\"rate-test-model-c8412b-kserve-self-signed-certs\""
Jul 09 19:38:30.673228 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.673207    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"]
Jul 09 19:38:30.719074 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.719041    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-home\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.719219 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.719079    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmmnf\" (UniqueName: \"kubernetes.io/projected/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-kube-api-access-xmmnf\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.719219 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.719180    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-dshm\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.719291 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.719228    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-model-cache\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.719291 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.719253    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tmp-dir\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.719291 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.719274    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tls-certs\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.820589 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.820550    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-dshm\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.820589 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.820593    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-model-cache\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.820857 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.820615    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tmp-dir\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.820857 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.820639    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tls-certs\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.820857 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.820673    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-home\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.820857 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.820722    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmmnf\" (UniqueName: \"kubernetes.io/projected/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-kube-api-access-xmmnf\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.821059 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.820992    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-model-cache\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.821059 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.821049    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tmp-dir\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.821138 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.821125    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-home\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.823102 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.823083    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-dshm\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.823214 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.823198    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tls-certs\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.830246 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.830227    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmmnf\" (UniqueName: \"kubernetes.io/projected/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-kube-api-access-xmmnf\") pod \"rate-test-model-c8412b-kserve-78bc45f8f9-25rlg\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") " pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:30.975785 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:30.975668    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:31.117547 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:31.117506    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"]
Jul 09 19:38:31.118511 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:38:31.118486    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd20abfc8_ecec_45c2_8d1b_c73a45410b8f.slice/crio-f680ed5da7570b8443742878e60315656345eaf24b365abf74d65042781e4b8e WatchSource:0}: Error finding container f680ed5da7570b8443742878e60315656345eaf24b365abf74d65042781e4b8e: Status 404 returned error can't find the container with id f680ed5da7570b8443742878e60315656345eaf24b365abf74d65042781e4b8e
Jul 09 19:38:31.721335 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:31.721293    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg" event={"ID":"d20abfc8-ecec-45c2-8d1b-c73a45410b8f","Type":"ContainerStarted","Data":"8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60"}
Jul 09 19:38:31.721335 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:31.721340    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg" event={"ID":"d20abfc8-ecec-45c2-8d1b-c73a45410b8f","Type":"ContainerStarted","Data":"f680ed5da7570b8443742878e60315656345eaf24b365abf74d65042781e4b8e"}
Jul 09 19:38:31.740617 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:31.740571    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg" podStartSLOduration=1.740557714 podStartE2EDuration="1.740557714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-09 19:38:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:38:31.738598498 +0000 UTC m=+2354.322030677" watchObservedRunningTime="2026-07-09 19:38:31.740557714 +0000 UTC m=+2354.323989894"
Jul 09 19:38:31.752224 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:31.752193    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2" podUID="18d48980-2e73-4bad-af49-c40d46402c3f" containerName="main" containerID="cri-o://30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4" gracePeriod=45
Jul 09 19:38:32.019847 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.019822    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:38:32.031709 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.031661    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18d48980-2e73-4bad-af49-c40d46402c3f-tls-certs\") pod \"18d48980-2e73-4bad-af49-c40d46402c3f\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") "
Jul 09 19:38:32.031867 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.031718    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-tmp-dir\") pod \"18d48980-2e73-4bad-af49-c40d46402c3f\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") "
Jul 09 19:38:32.031867 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.031754    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfs7s\" (UniqueName: \"kubernetes.io/projected/18d48980-2e73-4bad-af49-c40d46402c3f-kube-api-access-qfs7s\") pod \"18d48980-2e73-4bad-af49-c40d46402c3f\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") "
Jul 09 19:38:32.031867 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.031808    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-model-cache\") pod \"18d48980-2e73-4bad-af49-c40d46402c3f\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") "
Jul 09 19:38:32.031867 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.031833    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-home\") pod \"18d48980-2e73-4bad-af49-c40d46402c3f\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") "
Jul 09 19:38:32.032068 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.031934    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-dshm\") pod \"18d48980-2e73-4bad-af49-c40d46402c3f\" (UID: \"18d48980-2e73-4bad-af49-c40d46402c3f\") "
Jul 09 19:38:32.032068 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.032037    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18d48980-2e73-4bad-af49-c40d46402c3f" (UID: "18d48980-2e73-4bad-af49-c40d46402c3f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:38:32.032166 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.032142    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-home" (OuterVolumeSpecName: "home") pod "18d48980-2e73-4bad-af49-c40d46402c3f" (UID: "18d48980-2e73-4bad-af49-c40d46402c3f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:38:32.032221 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.032156    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-model-cache" (OuterVolumeSpecName: "model-cache") pod "18d48980-2e73-4bad-af49-c40d46402c3f" (UID: "18d48980-2e73-4bad-af49-c40d46402c3f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:38:32.032221 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.032205    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-tmp-dir\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:38:32.032300 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.032223    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-home\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:38:32.035046 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.035022    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-dshm" (OuterVolumeSpecName: "dshm") pod "18d48980-2e73-4bad-af49-c40d46402c3f" (UID: "18d48980-2e73-4bad-af49-c40d46402c3f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:38:32.035157 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.035017    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18d48980-2e73-4bad-af49-c40d46402c3f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18d48980-2e73-4bad-af49-c40d46402c3f" (UID: "18d48980-2e73-4bad-af49-c40d46402c3f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:38:32.035157 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.035108    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18d48980-2e73-4bad-af49-c40d46402c3f-kube-api-access-qfs7s" (OuterVolumeSpecName: "kube-api-access-qfs7s") pod "18d48980-2e73-4bad-af49-c40d46402c3f" (UID: "18d48980-2e73-4bad-af49-c40d46402c3f"). InnerVolumeSpecName "kube-api-access-qfs7s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:38:32.133361 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.133332    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-dshm\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:38:32.133361 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.133358    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18d48980-2e73-4bad-af49-c40d46402c3f-tls-certs\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:38:32.133361 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.133369    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfs7s\" (UniqueName: \"kubernetes.io/projected/18d48980-2e73-4bad-af49-c40d46402c3f-kube-api-access-qfs7s\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:38:32.133573 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.133377    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18d48980-2e73-4bad-af49-c40d46402c3f-model-cache\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:38:32.728149 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.728113    2569 generic.go:356] "Generic (PLEG): container finished" podID="18d48980-2e73-4bad-af49-c40d46402c3f" containerID="30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4" exitCode=0
Jul 09 19:38:32.728600 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.728175    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"
Jul 09 19:38:32.728600 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.728200    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2" event={"ID":"18d48980-2e73-4bad-af49-c40d46402c3f","Type":"ContainerDied","Data":"30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4"}
Jul 09 19:38:32.728600 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.728255    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2" event={"ID":"18d48980-2e73-4bad-af49-c40d46402c3f","Type":"ContainerDied","Data":"932b4165f21fa273fe157c7decb1180cac546dd406b907f8e6ca6ed1e26e3531"}
Jul 09 19:38:32.728600 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.728276    2569 scope.go:117] "RemoveContainer" containerID="30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4"
Jul 09 19:38:32.740345 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.740322    2569 scope.go:117] "RemoveContainer" containerID="30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4"
Jul 09 19:38:32.740596 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:38:32.740576    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4\": container with ID starting with 30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4 not found: ID does not exist" containerID="30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4"
Jul 09 19:38:32.740665 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.740604    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4"} err="failed to get container status \"30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4\": rpc error: code = NotFound desc = could not find container \"30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4\": container with ID starting with 30fdc7f8a340f710ddb689f41b5a77ab8bb4698c5bcc9f12325539006d7840d4 not found: ID does not exist"
Jul 09 19:38:32.751664 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.751643    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"]
Jul 09 19:38:32.755407 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:32.755377    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-a94244/sub-test-model-a94244-kserve-5975ddc9d7-z9jk2"]
Jul 09 19:38:33.878052 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:33.878024    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18d48980-2e73-4bad-af49-c40d46402c3f" path="/var/lib/kubelet/pods/18d48980-2e73-4bad-af49-c40d46402c3f/volumes"
Jul 09 19:38:40.976017 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:40.975971    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:40.976416 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:40.976281    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:40.992188 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:40.992158    2569 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:38:41.790357 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:38:41.790329    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:39:40.147261 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:40.147228    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"]
Jul 09 19:39:44.597731 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.597697    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7365-139637667022896"]
Jul 09 19:39:44.598150 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.598136    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18d48980-2e73-4bad-af49-c40d46402c3f" containerName="main"
Jul 09 19:39:44.598193 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.598152    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="18d48980-2e73-4bad-af49-c40d46402c3f" containerName="main"
Jul 09 19:39:44.598261 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.598251    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="18d48980-2e73-4bad-af49-c40d46402c3f" containerName="main"
Jul 09 19:39:44.601621 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.601600    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637667022896"
Jul 09 19:39:44.604437 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.604411    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-xmm4p\""
Jul 09 19:39:44.604701 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.604666    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7365-139637667022896"]
Jul 09 19:39:44.659205 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.659173    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn6gv\" (UniqueName: \"kubernetes.io/projected/cf0ccccd-f1af-428f-b516-77cbb8de10cb-kube-api-access-xn6gv\") pod \"test-curl-7365-139637667022896\" (UID: \"cf0ccccd-f1af-428f-b516-77cbb8de10cb\") " pod="openshift-ingress/test-curl-7365-139637667022896"
Jul 09 19:39:44.760489 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.760460    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn6gv\" (UniqueName: \"kubernetes.io/projected/cf0ccccd-f1af-428f-b516-77cbb8de10cb-kube-api-access-xn6gv\") pod \"test-curl-7365-139637667022896\" (UID: \"cf0ccccd-f1af-428f-b516-77cbb8de10cb\") " pod="openshift-ingress/test-curl-7365-139637667022896"
Jul 09 19:39:44.769331 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.769306    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn6gv\" (UniqueName: \"kubernetes.io/projected/cf0ccccd-f1af-428f-b516-77cbb8de10cb-kube-api-access-xn6gv\") pod \"test-curl-7365-139637667022896\" (UID: \"cf0ccccd-f1af-428f-b516-77cbb8de10cb\") " pod="openshift-ingress/test-curl-7365-139637667022896"
Jul 09 19:39:44.911846 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:44.911813    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637667022896"
Jul 09 19:39:45.054169 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:45.054142    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7365-139637667022896"]
Jul 09 19:39:45.056203 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:39:45.056177    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf0ccccd_f1af_428f_b516_77cbb8de10cb.slice/crio-f835b26765647ef8c38d2d7f34c7130da89bebbe6e645e7bf3dbfd0586a62f09 WatchSource:0}: Error finding container f835b26765647ef8c38d2d7f34c7130da89bebbe6e645e7bf3dbfd0586a62f09: Status 404 returned error can't find the container with id f835b26765647ef8c38d2d7f34c7130da89bebbe6e645e7bf3dbfd0586a62f09
Jul 09 19:39:45.057869 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:45.057854    2569 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 09 19:39:45.134513 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:45.134479    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7365-139637667022896" event={"ID":"cf0ccccd-f1af-428f-b516-77cbb8de10cb","Type":"ContainerStarted","Data":"f835b26765647ef8c38d2d7f34c7130da89bebbe6e645e7bf3dbfd0586a62f09"}
Jul 09 19:39:45.967198 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:39:45.967167    2569 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf0ccccd_f1af_428f_b516_77cbb8de10cb.slice/crio-7cd3eb3537383a5071a0709d4535a92df901acace1eae3b75f2e88a959e1da24.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcf0ccccd_f1af_428f_b516_77cbb8de10cb.slice/crio-conmon-7cd3eb3537383a5071a0709d4535a92df901acace1eae3b75f2e88a959e1da24.scope\": RecentStats: unable to find data in memory cache]"
Jul 09 19:39:46.141123 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:46.141084    2569 generic.go:356] "Generic (PLEG): container finished" podID="cf0ccccd-f1af-428f-b516-77cbb8de10cb" containerID="7cd3eb3537383a5071a0709d4535a92df901acace1eae3b75f2e88a959e1da24" exitCode=0
Jul 09 19:39:46.141288 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:46.141167    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7365-139637667022896" event={"ID":"cf0ccccd-f1af-428f-b516-77cbb8de10cb","Type":"ContainerDied","Data":"7cd3eb3537383a5071a0709d4535a92df901acace1eae3b75f2e88a959e1da24"}
Jul 09 19:39:47.298145 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.298116    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637667022896"
Jul 09 19:39:47.336605 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.336577    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7365-139637667022896_cf0ccccd-f1af-428f-b516-77cbb8de10cb/test-curl-7365-139637667022896/0.log"
Jul 09 19:39:47.361821 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.361787    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7365-139637667022896"]
Jul 09 19:39:47.365113 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.365092    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7365-139637667022896"]
Jul 09 19:39:47.382919 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.382897    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xn6gv\" (UniqueName: \"kubernetes.io/projected/cf0ccccd-f1af-428f-b516-77cbb8de10cb-kube-api-access-xn6gv\") pod \"cf0ccccd-f1af-428f-b516-77cbb8de10cb\" (UID: \"cf0ccccd-f1af-428f-b516-77cbb8de10cb\") "
Jul 09 19:39:47.385044 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.385022    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf0ccccd-f1af-428f-b516-77cbb8de10cb-kube-api-access-xn6gv" (OuterVolumeSpecName: "kube-api-access-xn6gv") pod "cf0ccccd-f1af-428f-b516-77cbb8de10cb" (UID: "cf0ccccd-f1af-428f-b516-77cbb8de10cb"). InnerVolumeSpecName "kube-api-access-xn6gv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:39:47.484409 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.484329    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xn6gv\" (UniqueName: \"kubernetes.io/projected/cf0ccccd-f1af-428f-b516-77cbb8de10cb-kube-api-access-xn6gv\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:47.878214 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:47.878185    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf0ccccd-f1af-428f-b516-77cbb8de10cb" path="/var/lib/kubelet/pods/cf0ccccd-f1af-428f-b516-77cbb8de10cb/volumes"
Jul 09 19:39:48.154620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:48.154539    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637667022896"
Jul 09 19:39:48.154620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:48.154597    2569 scope.go:117] "RemoveContainer" containerID="7cd3eb3537383a5071a0709d4535a92df901acace1eae3b75f2e88a959e1da24"
Jul 09 19:39:50.632294 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.632257    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7365-139637668018048"]
Jul 09 19:39:50.632674 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.632663    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf0ccccd-f1af-428f-b516-77cbb8de10cb" containerName="test-curl-7365-139637667022896"
Jul 09 19:39:50.632740 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.632676    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf0ccccd-f1af-428f-b516-77cbb8de10cb" containerName="test-curl-7365-139637667022896"
Jul 09 19:39:50.632776 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.632770    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf0ccccd-f1af-428f-b516-77cbb8de10cb" containerName="test-curl-7365-139637667022896"
Jul 09 19:39:50.637301 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.637279    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637668018048"
Jul 09 19:39:50.638960 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.638933    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7365-139637668018048"]
Jul 09 19:39:50.640219 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.640201    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-xmm4p\""
Jul 09 19:39:50.714660 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.714631    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8cqr\" (UniqueName: \"kubernetes.io/projected/c6776660-4364-440e-85e0-9f7f3695ad42-kube-api-access-r8cqr\") pod \"test-curl-7365-139637668018048\" (UID: \"c6776660-4364-440e-85e0-9f7f3695ad42\") " pod="openshift-ingress/test-curl-7365-139637668018048"
Jul 09 19:39:50.815751 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.815715    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8cqr\" (UniqueName: \"kubernetes.io/projected/c6776660-4364-440e-85e0-9f7f3695ad42-kube-api-access-r8cqr\") pod \"test-curl-7365-139637668018048\" (UID: \"c6776660-4364-440e-85e0-9f7f3695ad42\") " pod="openshift-ingress/test-curl-7365-139637668018048"
Jul 09 19:39:50.824862 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.824830    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8cqr\" (UniqueName: \"kubernetes.io/projected/c6776660-4364-440e-85e0-9f7f3695ad42-kube-api-access-r8cqr\") pod \"test-curl-7365-139637668018048\" (UID: \"c6776660-4364-440e-85e0-9f7f3695ad42\") " pod="openshift-ingress/test-curl-7365-139637668018048"
Jul 09 19:39:50.947752 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:50.947647    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637668018048"
Jul 09 19:39:51.087489 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:51.087457    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7365-139637668018048"]
Jul 09 19:39:51.087888 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:39:51.087859    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6776660_4364_440e_85e0_9f7f3695ad42.slice/crio-35fe84c62c1e6390de61635d81f6c704f48ce0e12fd67a36ad249164eb0dbb53 WatchSource:0}: Error finding container 35fe84c62c1e6390de61635d81f6c704f48ce0e12fd67a36ad249164eb0dbb53: Status 404 returned error can't find the container with id 35fe84c62c1e6390de61635d81f6c704f48ce0e12fd67a36ad249164eb0dbb53
Jul 09 19:39:51.174053 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:51.174027    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7365-139637668018048" event={"ID":"c6776660-4364-440e-85e0-9f7f3695ad42","Type":"ContainerStarted","Data":"35fe84c62c1e6390de61635d81f6c704f48ce0e12fd67a36ad249164eb0dbb53"}
Jul 09 19:39:52.182400 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:52.182363    2569 generic.go:356] "Generic (PLEG): container finished" podID="c6776660-4364-440e-85e0-9f7f3695ad42" containerID="c455f4d798ad499f217e1e359ca45190cb8491e3f130b58f3db234432b8043bf" exitCode=0
Jul 09 19:39:52.182801 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:52.182444    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7365-139637668018048" event={"ID":"c6776660-4364-440e-85e0-9f7f3695ad42","Type":"ContainerDied","Data":"c455f4d798ad499f217e1e359ca45190cb8491e3f130b58f3db234432b8043bf"}
Jul 09 19:39:53.338482 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.338457    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637668018048"
Jul 09 19:39:53.359049 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.359025    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7365-139637668018048_c6776660-4364-440e-85e0-9f7f3695ad42/test-curl-7365-139637668018048/0.log"
Jul 09 19:39:53.383464 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.383430    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7365-139637668018048"]
Jul 09 19:39:53.386842 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.386819    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7365-139637668018048"]
Jul 09 19:39:53.439320 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.439292    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r8cqr\" (UniqueName: \"kubernetes.io/projected/c6776660-4364-440e-85e0-9f7f3695ad42-kube-api-access-r8cqr\") pod \"c6776660-4364-440e-85e0-9f7f3695ad42\" (UID: \"c6776660-4364-440e-85e0-9f7f3695ad42\") "
Jul 09 19:39:53.441360 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.441337    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6776660-4364-440e-85e0-9f7f3695ad42-kube-api-access-r8cqr" (OuterVolumeSpecName: "kube-api-access-r8cqr") pod "c6776660-4364-440e-85e0-9f7f3695ad42" (UID: "c6776660-4364-440e-85e0-9f7f3695ad42"). InnerVolumeSpecName "kube-api-access-r8cqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:39:53.539971 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.539941    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r8cqr\" (UniqueName: \"kubernetes.io/projected/c6776660-4364-440e-85e0-9f7f3695ad42-kube-api-access-r8cqr\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:53.877904 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:53.877873    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6776660-4364-440e-85e0-9f7f3695ad42" path="/var/lib/kubelet/pods/c6776660-4364-440e-85e0-9f7f3695ad42/volumes"
Jul 09 19:39:54.200228 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:54.200140    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7365-139637668018048"
Jul 09 19:39:54.200228 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:54.200180    2569 scope.go:117] "RemoveContainer" containerID="c455f4d798ad499f217e1e359ca45190cb8491e3f130b58f3db234432b8043bf"
Jul 09 19:39:55.168124 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.168089    2569 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg" podUID="d20abfc8-ecec-45c2-8d1b-c73a45410b8f" containerName="main" containerID="cri-o://8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60" gracePeriod=45
Jul 09 19:39:55.432173 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.432117    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:39:55.557784 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.557754    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-dshm\") pod \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") "
Jul 09 19:39:55.557943 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.557799    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-home\") pod \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") "
Jul 09 19:39:55.557943 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.557816    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tmp-dir\") pod \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") "
Jul 09 19:39:55.557943 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.557856    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmmnf\" (UniqueName: \"kubernetes.io/projected/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-kube-api-access-xmmnf\") pod \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") "
Jul 09 19:39:55.557943 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.557873    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tls-certs\") pod \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") "
Jul 09 19:39:55.557943 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.557937    2569 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-model-cache\") pod \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\" (UID: \"d20abfc8-ecec-45c2-8d1b-c73a45410b8f\") "
Jul 09 19:39:55.558183 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.558034    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-home" (OuterVolumeSpecName: "home") pod "d20abfc8-ecec-45c2-8d1b-c73a45410b8f" (UID: "d20abfc8-ecec-45c2-8d1b-c73a45410b8f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:39:55.558183 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.558112    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d20abfc8-ecec-45c2-8d1b-c73a45410b8f" (UID: "d20abfc8-ecec-45c2-8d1b-c73a45410b8f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:39:55.558286 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.558211    2569 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-home\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:55.558286 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.558226    2569 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tmp-dir\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:55.558286 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.558273    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-model-cache" (OuterVolumeSpecName: "model-cache") pod "d20abfc8-ecec-45c2-8d1b-c73a45410b8f" (UID: "d20abfc8-ecec-45c2-8d1b-c73a45410b8f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:39:55.560107 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.560080    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-dshm" (OuterVolumeSpecName: "dshm") pod "d20abfc8-ecec-45c2-8d1b-c73a45410b8f" (UID: "d20abfc8-ecec-45c2-8d1b-c73a45410b8f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 09 19:39:55.560231 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.560108    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d20abfc8-ecec-45c2-8d1b-c73a45410b8f" (UID: "d20abfc8-ecec-45c2-8d1b-c73a45410b8f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 09 19:39:55.560231 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.560219    2569 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-kube-api-access-xmmnf" (OuterVolumeSpecName: "kube-api-access-xmmnf") pod "d20abfc8-ecec-45c2-8d1b-c73a45410b8f" (UID: "d20abfc8-ecec-45c2-8d1b-c73a45410b8f"). InnerVolumeSpecName "kube-api-access-xmmnf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 09 19:39:55.658891 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.658851    2569 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-model-cache\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:55.658891 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.658886    2569 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-dshm\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:55.659049 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.658900    2569 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmmnf\" (UniqueName: \"kubernetes.io/projected/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-kube-api-access-xmmnf\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:55.659049 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:55.658913    2569 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d20abfc8-ecec-45c2-8d1b-c73a45410b8f-tls-certs\") on node \"ip-10-0-128-157.ec2.internal\" DevicePath \"\""
Jul 09 19:39:56.213861 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.213779    2569 generic.go:356] "Generic (PLEG): container finished" podID="d20abfc8-ecec-45c2-8d1b-c73a45410b8f" containerID="8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60" exitCode=0
Jul 09 19:39:56.213861 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.213818    2569 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"
Jul 09 19:39:56.214288 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.213868    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg" event={"ID":"d20abfc8-ecec-45c2-8d1b-c73a45410b8f","Type":"ContainerDied","Data":"8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60"}
Jul 09 19:39:56.214288 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.213906    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg" event={"ID":"d20abfc8-ecec-45c2-8d1b-c73a45410b8f","Type":"ContainerDied","Data":"f680ed5da7570b8443742878e60315656345eaf24b365abf74d65042781e4b8e"}
Jul 09 19:39:56.214288 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.213927    2569 scope.go:117] "RemoveContainer" containerID="8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60"
Jul 09 19:39:56.222925 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.222908    2569 scope.go:117] "RemoveContainer" containerID="8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60"
Jul 09 19:39:56.223178 ip-10-0-128-157 kubenswrapper[2569]: E0709 19:39:56.223160    2569 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60\": container with ID starting with 8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60 not found: ID does not exist" containerID="8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60"
Jul 09 19:39:56.223243 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.223190    2569 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60"} err="failed to get container status \"8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60\": rpc error: code = NotFound desc = could not find container \"8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60\": container with ID starting with 8656b27afe995958fd896b182b90517631f7ed95e3b133b23c63210369be8a60 not found: ID does not exist"
Jul 09 19:39:56.234471 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.234449    2569 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"]
Jul 09 19:39:56.240754 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:56.240732    2569 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c8412b/rate-test-model-c8412b-kserve-78bc45f8f9-25rlg"]
Jul 09 19:39:57.878550 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:39:57.878516    2569 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d20abfc8-ecec-45c2-8d1b-c73a45410b8f" path="/var/lib/kubelet/pods/d20abfc8-ecec-45c2-8d1b-c73a45410b8f/volumes"
Jul 09 19:40:13.984416 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:13.984383    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8b44b5c9b-q6n8f_331e6bc2-9bf9-4f5f-b039-38d480d78378/authorino/0.log"
Jul 09 19:40:18.702664 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:18.702628    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lgpkh_8273d8a6-ed4f-4d63-be7d-1b49690a8741/manager/0.log"
Jul 09 19:40:18.820203 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:18.820175    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4jfjt_acddc5e0-b901-418b-88a2-56dfc9705a3a/manager/0.log"
Jul 09 19:40:18.940523 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:18.940494    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75c66f9f7-sz7zm_435bd819-2d30-4e95-af9d-842eef51b157/manager/0.log"
Jul 09 19:40:19.055742 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:19.055657    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rbj5f_2c702c73-fae0-4b1f-8886-4577e75cc21f/server/0.log"
Jul 09 19:40:19.171719 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:19.171674    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lv5bf_cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3/manager/0.log"
Jul 09 19:40:19.289881 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:19.289855    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779db88bb-7fl8f_c82727eb-817e-4415-9b10-9b8b0d6c8803/copy-manifests/0.log"
Jul 09 19:40:19.303737 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:19.303702    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779db88bb-7fl8f_c82727eb-817e-4415-9b10-9b8b0d6c8803/manager/0.log"
Jul 09 19:40:19.902168 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:19.902138    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-67f4cb5d58-swlsk_f25fad68-c8b2-4e37-ac5c-5fa1dc187f6f/maas-api/0.log"
Jul 09 19:40:20.017153 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:20.017128    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-6gjf9_88e1dc59-5cd2-4cae-852e-4ded32e4261e/postgres/0.log"
Jul 09 19:40:20.790304 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:20.790279    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m_1d6ff828-0473-4a21-a38f-0164ca5b4d9a/util/0.log"
Jul 09 19:40:20.797437 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:20.797408    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m_1d6ff828-0473-4a21-a38f-0164ca5b4d9a/pull/0.log"
Jul 09 19:40:20.804426 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:20.804394    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m_1d6ff828-0473-4a21-a38f-0164ca5b4d9a/extract/0.log"
Jul 09 19:40:20.920445 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:20.920414    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts_df2c15b7-0dbe-4fee-adea-df55bfbb74b7/util/0.log"
Jul 09 19:40:20.932313 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:20.932278    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts_df2c15b7-0dbe-4fee-adea-df55bfbb74b7/pull/0.log"
Jul 09 19:40:20.939054 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:20.939034    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts_df2c15b7-0dbe-4fee-adea-df55bfbb74b7/extract/0.log"
Jul 09 19:40:21.046649 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.046581    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f_7b10431a-04b9-4b7c-909d-7d67bf556c50/util/0.log"
Jul 09 19:40:21.052760 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.052739    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f_7b10431a-04b9-4b7c-909d-7d67bf556c50/pull/0.log"
Jul 09 19:40:21.059190 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.059159    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f_7b10431a-04b9-4b7c-909d-7d67bf556c50/extract/0.log"
Jul 09 19:40:21.174915 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.174883    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8_9d237620-7db5-4607-b1c1-a9d054e3c05b/util/0.log"
Jul 09 19:40:21.182434 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.182413    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8_9d237620-7db5-4607-b1c1-a9d054e3c05b/pull/0.log"
Jul 09 19:40:21.189709 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.189667    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8_9d237620-7db5-4607-b1c1-a9d054e3c05b/extract/0.log"
Jul 09 19:40:21.303744 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.303641    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8b44b5c9b-q6n8f_331e6bc2-9bf9-4f5f-b039-38d480d78378/authorino/0.log"
Jul 09 19:40:21.420021 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.419990    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zbgfv_d58e8cbc-b7ed-4a2b-b125-80d72c2c6058/manager/0.log"
Jul 09 19:40:21.535169 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.535139    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-whw9v_138c1a2e-aca2-4c06-8625-8b4bc25f28ad/manager/0.log"
Jul 09 19:40:21.778984 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.778953    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hklx6_1e5e329a-afaa-41cc-b0d4-7c3b27ff4826/registry-server/0.log"
Jul 09 19:40:21.899636 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:21.899604    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bnqcn_98f45300-a79c-4b63-bc9c-cfdf2a2c2b92/manager/0.log"
Jul 09 19:40:22.129756 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:22.129727    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cmtbn_9c67e2be-32aa-4fcf-be19-dae79fe74dc2/manager/0.log"
Jul 09 19:40:22.816521 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:22.816485    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6c4d4-jdvzz_4543215d-3a28-44f5-803d-014eb3ebbc9f/kube-auth-proxy/0.log"
Jul 09 19:40:23.047813 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:23.047787    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qcf4q_9d34a207-f5f3-418f-ac31-b4dbdeee2ec2/payload-pre-processing/0.log"
Jul 09 19:40:23.165605 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:23.165575    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lbzgm_8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0/payload-processing/0.log"
Jul 09 19:40:23.282388 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:23.282356    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b7779465d-kmdjx_c8467136-f179-4e60-8a06-c6c12d69e475/router/0.log"
Jul 09 19:40:23.727312 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:23.727279    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-rv2x9_d3e4344b-9d79-4046-8dea-4698a69b145d/main/0.log"
Jul 09 19:40:23.956177 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:23.956155    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86db5vt8q_e8704231-8e17-4bd6-8782-d5fc80ad613f/main/0.log"
Jul 09 19:40:30.523336 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:30.523303    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9tnc2_ae2e2595-c6c5-47e7-88f9-1661b5a3c4fc/global-pull-secret-syncer/0.log"
Jul 09 19:40:30.741242 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:30.741214    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x48nr_08bf4c59-eb78-4b76-ac07-eefab77990bd/konnectivity-agent/0.log"
Jul 09 19:40:30.762728 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:30.762699    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-157.ec2.internal_dcf208c2c2ddf8d2ac591147d73def93/haproxy/0.log"
Jul 09 19:40:34.429067 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.429041    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m_1d6ff828-0473-4a21-a38f-0164ca5b4d9a/extract/0.log"
Jul 09 19:40:34.457131 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.457111    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m_1d6ff828-0473-4a21-a38f-0164ca5b4d9a/util/0.log"
Jul 09 19:40:34.478430 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.478405    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b75969l4m_1d6ff828-0473-4a21-a38f-0164ca5b4d9a/pull/0.log"
Jul 09 19:40:34.544121 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.544106    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts_df2c15b7-0dbe-4fee-adea-df55bfbb74b7/extract/0.log"
Jul 09 19:40:34.569818 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.569801    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts_df2c15b7-0dbe-4fee-adea-df55bfbb74b7/util/0.log"
Jul 09 19:40:34.612658 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.612632    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04nfts_df2c15b7-0dbe-4fee-adea-df55bfbb74b7/pull/0.log"
Jul 09 19:40:34.649358 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.649333    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f_7b10431a-04b9-4b7c-909d-7d67bf556c50/extract/0.log"
Jul 09 19:40:34.671044 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.671025    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f_7b10431a-04b9-4b7c-909d-7d67bf556c50/util/0.log"
Jul 09 19:40:34.692807 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.692754    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73bv92f_7b10431a-04b9-4b7c-909d-7d67bf556c50/pull/0.log"
Jul 09 19:40:34.718571 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.718552    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8_9d237620-7db5-4607-b1c1-a9d054e3c05b/extract/0.log"
Jul 09 19:40:34.739932 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.739911    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8_9d237620-7db5-4607-b1c1-a9d054e3c05b/util/0.log"
Jul 09 19:40:34.761780 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:34.761760    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1rvrm8_9d237620-7db5-4607-b1c1-a9d054e3c05b/pull/0.log"
Jul 09 19:40:35.036993 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:35.036913    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-8b44b5c9b-q6n8f_331e6bc2-9bf9-4f5f-b039-38d480d78378/authorino/0.log"
Jul 09 19:40:35.084588 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:35.084563    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-zbgfv_d58e8cbc-b7ed-4a2b-b125-80d72c2c6058/manager/0.log"
Jul 09 19:40:35.119887 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:35.119863    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-whw9v_138c1a2e-aca2-4c06-8625-8b4bc25f28ad/manager/0.log"
Jul 09 19:40:35.210867 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:35.210838    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hklx6_1e5e329a-afaa-41cc-b0d4-7c3b27ff4826/registry-server/0.log"
Jul 09 19:40:35.346658 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:35.346630    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-bnqcn_98f45300-a79c-4b63-bc9c-cfdf2a2c2b92/manager/0.log"
Jul 09 19:40:35.453871 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:35.453842    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-cmtbn_9c67e2be-32aa-4fcf-be19-dae79fe74dc2/manager/0.log"
Jul 09 19:40:37.314783 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:37.314750    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-5b4kv_5668efae-c36d-43d1-8c85-4af4d19c1d3e/cluster-monitoring-operator/0.log"
Jul 09 19:40:37.596838 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:37.596758    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n6qfs_f9bd1049-aec4-4152-b132-4ea86a21dd7e/node-exporter/0.log"
Jul 09 19:40:37.618558 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:37.618538    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n6qfs_f9bd1049-aec4-4152-b132-4ea86a21dd7e/kube-rbac-proxy/0.log"
Jul 09 19:40:37.644392 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:37.644373    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-n6qfs_f9bd1049-aec4-4152-b132-4ea86a21dd7e/init-textfile/0.log"
Jul 09 19:40:39.827861 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:39.827825    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6cf856758c-rgg8w_72ca0d0e-77d2-4516-8a52-da3a36a40f1e/networking-console-plugin/0.log"
Jul 09 19:40:41.025462 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.025429    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-gfwmb_8c720f71-d20e-4dac-b09f-c8100d4b3ffb/download-server/0.log"
Jul 09 19:40:41.117974 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.117943    2569 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"]
Jul 09 19:40:41.118336 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.118323    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6776660-4364-440e-85e0-9f7f3695ad42" containerName="test-curl-7365-139637668018048"
Jul 09 19:40:41.118376 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.118338    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6776660-4364-440e-85e0-9f7f3695ad42" containerName="test-curl-7365-139637668018048"
Jul 09 19:40:41.118376 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.118353    2569 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d20abfc8-ecec-45c2-8d1b-c73a45410b8f" containerName="main"
Jul 09 19:40:41.118376 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.118360    2569 state_mem.go:107] "Deleted CPUSet assignment" podUID="d20abfc8-ecec-45c2-8d1b-c73a45410b8f" containerName="main"
Jul 09 19:40:41.118471 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.118446    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="d20abfc8-ecec-45c2-8d1b-c73a45410b8f" containerName="main"
Jul 09 19:40:41.118471 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.118455    2569 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6776660-4364-440e-85e0-9f7f3695ad42" containerName="test-curl-7365-139637668018048"
Jul 09 19:40:41.121564 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.121549    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.124576 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.124555    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vtw49\"/\"openshift-service-ca.crt\""
Jul 09 19:40:41.126056 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.126031    2569 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vtw49\"/\"default-dockercfg-8sjz6\""
Jul 09 19:40:41.126173 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.126109    2569 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vtw49\"/\"kube-root-ca.crt\""
Jul 09 19:40:41.129297 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.129279    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"]
Jul 09 19:40:41.249751 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.249721    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-lib-modules\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.249910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.249762    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-sys\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.249910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.249781    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmvct\" (UniqueName: \"kubernetes.io/projected/a2422266-9b2a-4b87-9c18-40364cac96a1-kube-api-access-cmvct\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.249910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.249814    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-proc\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.249910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.249835    2569 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-podres\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350466 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350380    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-proc\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350466 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350418    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-podres\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350484    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-lib-modules\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350509    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-proc\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350512    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-sys\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350564    2569 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmvct\" (UniqueName: \"kubernetes.io/projected/a2422266-9b2a-4b87-9c18-40364cac96a1-kube-api-access-cmvct\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350566    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-sys\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350635    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-podres\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.350661 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.350643    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a2422266-9b2a-4b87-9c18-40364cac96a1-lib-modules\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.360592 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.360570    2569 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmvct\" (UniqueName: \"kubernetes.io/projected/a2422266-9b2a-4b87-9c18-40364cac96a1-kube-api-access-cmvct\") pod \"perf-node-gather-daemonset-fq8jf\" (UID: \"a2422266-9b2a-4b87-9c18-40364cac96a1\") " pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.432584 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.432553    2569 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:41.568101 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.568067    2569 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"]
Jul 09 19:40:41.569740 ip-10-0-128-157 kubenswrapper[2569]: W0709 19:40:41.569716    2569 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda2422266_9b2a_4b87_9c18_40364cac96a1.slice/crio-cd82bf751b628c352f9bc3eaea8dddceed9f00dd932f386a0852d302c1d98614 WatchSource:0}: Error finding container cd82bf751b628c352f9bc3eaea8dddceed9f00dd932f386a0852d302c1d98614: Status 404 returned error can't find the container with id cd82bf751b628c352f9bc3eaea8dddceed9f00dd932f386a0852d302c1d98614
Jul 09 19:40:41.787109 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:41.787083    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-g9698_7380ebb1-08f3-4f39-a0fd-ca95b0e1a047/volume-data-source-validator/0.log"
Jul 09 19:40:42.476481 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:42.476447    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf" event={"ID":"a2422266-9b2a-4b87-9c18-40364cac96a1","Type":"ContainerStarted","Data":"4ed530e66c569f8ac7e663c6ceb41ca536370491115c364ff6dd36b284e2c860"}
Jul 09 19:40:42.476910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:42.476502    2569 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf" event={"ID":"a2422266-9b2a-4b87-9c18-40364cac96a1","Type":"ContainerStarted","Data":"cd82bf751b628c352f9bc3eaea8dddceed9f00dd932f386a0852d302c1d98614"}
Jul 09 19:40:42.476910 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:42.476530    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:42.494621 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:42.494566    2569 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf" podStartSLOduration=1.4945513 podStartE2EDuration="1.4945513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-09 19:40:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-09 19:40:42.492448494 +0000 UTC m=+2485.075880686" watchObservedRunningTime="2026-07-09 19:40:42.4945513 +0000 UTC m=+2485.077983480"
Jul 09 19:40:42.819359 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:42.819290    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-27zql_64b89e42-7a99-4b6f-afcf-9f3315c8c84c/dns/0.log"
Jul 09 19:40:42.839676 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:42.839656    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-27zql_64b89e42-7a99-4b6f-afcf-9f3315c8c84c/kube-rbac-proxy/0.log"
Jul 09 19:40:42.983251 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:42.983232    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-f5bwn_5bb527e7-82f3-4a04-ac64-789566abc7ac/dns-node-resolver/0.log"
Jul 09 19:40:43.533131 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:43.533100    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-75f67cc456-6r46r_c964eefe-7516-4ef1-852c-52cf0440928f/registry/0.log"
Jul 09 19:40:43.554353 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:43.554334    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gftdt_9524d05b-67d4-4944-9403-1d5d9a9e7872/node-ca/0.log"
Jul 09 19:40:44.652093 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:44.652068    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6c4d4-jdvzz_4543215d-3a28-44f5-803d-014eb3ebbc9f/kube-auth-proxy/0.log"
Jul 09 19:40:44.743820 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:44.743787    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qcf4q_9d34a207-f5f3-418f-ac31-b4dbdeee2ec2/payload-pre-processing/0.log"
Jul 09 19:40:44.778294 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:44.778264    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lbzgm_8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0/payload-processing/0.log"
Jul 09 19:40:44.802913 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:44.802894    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b7779465d-kmdjx_c8467136-f179-4e60-8a06-c6c12d69e475/router/0.log"
Jul 09 19:40:45.480113 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:45.480085    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-hx5wk_51fc951b-adc4-4219-9e6c-0865ff0e8b61/serve-healthcheck-canary/0.log"
Jul 09 19:40:45.992590 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:45.992564    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mcr7_a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1/kube-rbac-proxy/0.log"
Jul 09 19:40:46.014119 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:46.014094    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mcr7_a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1/exporter/0.log"
Jul 09 19:40:46.036939 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:46.036924    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6mcr7_a5b84a61-b9ef-4ff2-a5a6-3ff4728125e1/extractor/0.log"
Jul 09 19:40:48.257641 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.257613    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-lgpkh_8273d8a6-ed4f-4d63-be7d-1b49690a8741/manager/0.log"
Jul 09 19:40:48.400187 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.400160    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-4jfjt_acddc5e0-b901-418b-88a2-56dfc9705a3a/manager/0.log"
Jul 09 19:40:48.493172 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.493129    2569 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vtw49/perf-node-gather-daemonset-fq8jf"
Jul 09 19:40:48.500588 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.500567    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-75c66f9f7-sz7zm_435bd819-2d30-4e95-af9d-842eef51b157/manager/0.log"
Jul 09 19:40:48.522238 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.522178    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-rbj5f_2c702c73-fae0-4b1f-8886-4577e75cc21f/server/0.log"
Jul 09 19:40:48.588831 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.588807    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-lv5bf_cf17df4d-4378-4cb8-ad20-6b4de3bf9ab3/manager/0.log"
Jul 09 19:40:48.651531 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.651503    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779db88bb-7fl8f_c82727eb-817e-4415-9b10-9b8b0d6c8803/manager/0.log"
Jul 09 19:40:48.670620 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:48.670599    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-779db88bb-7fl8f_c82727eb-817e-4415-9b10-9b8b0d6c8803/copy-manifests/0.log"
Jul 09 19:40:50.102128 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:50.102049    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kvckn_821de831-177e-4081-9484-aa4710829b1f/openshift-lws-operator/0.log"
Jul 09 19:40:51.764586 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:51.764557    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6c4d4-jdvzz_4543215d-3a28-44f5-803d-014eb3ebbc9f/kube-auth-proxy/0.log"
Jul 09 19:40:51.831843 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:51.831814    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qcf4q_9d34a207-f5f3-418f-ac31-b4dbdeee2ec2/payload-pre-processing/0.log"
Jul 09 19:40:51.849729 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:51.849705    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lbzgm_8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0/payload-processing/0.log"
Jul 09 19:40:51.861534 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:51.861514    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b7779465d-kmdjx_c8467136-f179-4e60-8a06-c6c12d69e475/router/0.log"
Jul 09 19:40:53.301578 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:53.301534    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7b6c4d4-jdvzz_4543215d-3a28-44f5-803d-014eb3ebbc9f/kube-auth-proxy/0.log"
Jul 09 19:40:53.372278 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:53.372244    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-qcf4q_9d34a207-f5f3-418f-ac31-b4dbdeee2ec2/payload-pre-processing/0.log"
Jul 09 19:40:53.394363 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:53.394341    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-lbzgm_8beec3a6-4d6e-400c-8c1b-95b0b8f7bac0/payload-processing/0.log"
Jul 09 19:40:53.412166 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:53.412139    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6b7779465d-kmdjx_c8467136-f179-4e60-8a06-c6c12d69e475/router/0.log"
Jul 09 19:40:55.691512 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:55.691482    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-q6nxh_d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b/kube-storage-version-migrator-operator/1.log"
Jul 09 19:40:55.692271 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:55.692252    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-799cb7f958-q6nxh_d1d13420-7ca3-4bb9-92bc-8c9c3ec5369b/kube-storage-version-migrator-operator/0.log"
Jul 09 19:40:56.723159 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.723127    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-6z8dd_20730733-2d51-4667-acf3-374120aa43c5/kube-multus/0.log"
Jul 09 19:40:56.750522 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.750496    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tmrs_feeda688-dbc8-4a69-8c1c-e7487aad15ba/kube-multus-additional-cni-plugins/0.log"
Jul 09 19:40:56.770198 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.770177    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tmrs_feeda688-dbc8-4a69-8c1c-e7487aad15ba/egress-router-binary-copy/0.log"
Jul 09 19:40:56.792894 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.792871    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tmrs_feeda688-dbc8-4a69-8c1c-e7487aad15ba/cni-plugins/0.log"
Jul 09 19:40:56.816136 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.816103    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tmrs_feeda688-dbc8-4a69-8c1c-e7487aad15ba/bond-cni-plugin/0.log"
Jul 09 19:40:56.844385 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.844365    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tmrs_feeda688-dbc8-4a69-8c1c-e7487aad15ba/routeoverride-cni/0.log"
Jul 09 19:40:56.869465 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.869445    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tmrs_feeda688-dbc8-4a69-8c1c-e7487aad15ba/whereabouts-cni-bincopy/0.log"
Jul 09 19:40:56.891832 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:56.891807    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-6tmrs_feeda688-dbc8-4a69-8c1c-e7487aad15ba/whereabouts-cni/0.log"
Jul 09 19:40:57.339861 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:57.339834    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-srwqt_503853d6-c3ef-4994-9151-a4517a4c0abb/network-metrics-daemon/0.log"
Jul 09 19:40:57.357484 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:57.357463    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-srwqt_503853d6-c3ef-4994-9151-a4517a4c0abb/kube-rbac-proxy/0.log"
Jul 09 19:40:58.724436 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.724403    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/ovn-controller/0.log"
Jul 09 19:40:58.741015 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.740995    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/ovn-acl-logging/0.log"
Jul 09 19:40:58.762706 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.762670    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/kube-rbac-proxy-node/0.log"
Jul 09 19:40:58.782978 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.782959    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/kube-rbac-proxy-ovn-metrics/0.log"
Jul 09 19:40:58.803106 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.803089    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/northd/0.log"
Jul 09 19:40:58.823275 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.823256    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/nbdb/0.log"
Jul 09 19:40:58.843032 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.843011    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/sbdb/0.log"
Jul 09 19:40:58.907983 ip-10-0-128-157 kubenswrapper[2569]: I0709 19:40:58.907955    2569 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-drt6n_2cfad3f0-1b7d-4ce4-8555-4b283f3f2f73/ovnkube-controller/0.log"