Jul 21 17:58:09.208689 ip-10-0-140-73 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 21 17:58:09.208702 ip-10-0-140-73 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 21 17:58:09.208712 ip-10-0-140-73 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 21 17:58:09.209033 ip-10-0-140-73 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 21 17:58:19.218800 ip-10-0-140-73 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 21 17:58:19.218820 ip-10-0-140-73 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 37d61168ba284ce6b9fe7940ed417c85 --
Jul 21 18:00:10.374970 ip-10-0-140-73 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.430565    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439036    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439049    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439053    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439058    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439063    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439067    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439070    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 18:00:12.450192 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439073    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439076    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439079    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439083    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439089    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439092    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439095    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439098    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439101    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439104    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439107    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439109    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439112    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439115    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439117    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439121    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439125    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439135    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439141    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439145    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 18:00:12.454434 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439150    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439153    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439156    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439161    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439166    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439169    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439171    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439175    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439177    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439183    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439186    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439189    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439191    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439194    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439199    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439202    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439205    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439208    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439211    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 18:00:12.455285 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439213    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439216    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439221    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439224    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439227    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439229    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439232    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439235    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439238    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439240    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439243    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439245    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439249    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439252    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439257    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439260    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439262    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439265    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439268    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439271    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 18:00:12.458301 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439273    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439276    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439279    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439283    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439290    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439295    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439302    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439306    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439309    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439311    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439314    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439317    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439319    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439322    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439325    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439327    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439330    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439333    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439335    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439340    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 18:00:12.459172 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439343    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439345    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439348    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439351    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439354    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439357    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439359    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439362    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439365    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439368    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439371    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439376    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.439378    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440579    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440587    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440613    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440616    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440620    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440623    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440626    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 18:00:12.462179 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440632    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440635    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440637    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440640    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440643    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440645    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440648    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440651    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440654    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440657    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440659    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440662    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440665    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440670    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440673    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440676    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440679    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440682    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440685    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440687    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 18:00:12.463023 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440690    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440693    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440695    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440699    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440702    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440708    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440711    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440713    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440716    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440719    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440722    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440725    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440727    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440730    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440733    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440735    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440738    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440741    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440747    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440750    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 18:00:12.467298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440752    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440757    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440761    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440766    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440774    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440778    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440781    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440783    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440786    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440789    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440794    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440797    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440800    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440803    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440806    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440808    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440812    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440817    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440821    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 18:00:12.468120 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440823    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440827    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440834    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440836    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440839    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440842    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440844    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440848    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440850    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440853    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440856    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440858    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440861    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440863    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440866    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440871    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440873    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440877    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440881    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 18:00:12.471580 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440884    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440889    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440893    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440896    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440898    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440901    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440904    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440907    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440912    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440915    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440919    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440926    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440932    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.440937    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441644    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441651    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441655    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441658    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441663    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 18:00:12.472421 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441674    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441678    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441682    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441686    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441689    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441692    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441695    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441701    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441704    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441707    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441709    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441712    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441714    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441717    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441722    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441725    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441728    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441730    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441734    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 18:00:12.475867 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441739    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441753    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441758    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441763    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441766    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441769    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441772    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441775    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441779    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441783    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441786    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441789    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441792    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441798    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441801    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441804    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441808    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441811    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441813    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441816    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 18:00:12.476697 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441819    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441821    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441824    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441827    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441829    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441835    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441837    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441842    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441844    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441847    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441850    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441853    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441855    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441858    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441862    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441865    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441870    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441873    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441876    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 18:00:12.582189 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441878    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441881    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441884    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441886    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441889    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441892    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441894    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441897    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441902    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441909    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441919    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441923    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441926    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441928    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441931    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441934    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441936    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441939    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441941    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441944    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 18:00:12.583030 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441948    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441951    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441954    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441959    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441961    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441964    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441966    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441969    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441972    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441975    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441977    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441980    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441983    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441985    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441988    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.441991    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442094    2578 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442102    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442111    2578 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442116    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442120    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 18:00:12.587308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442126    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442131    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442135    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442138    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442141    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442147    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442151    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442163    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442166    2578 flags.go:64] FLAG: --cgroup-root=""
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442170    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442173    2578 flags.go:64] FLAG: --client-ca-file=""
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442178    2578 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442181    2578 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442186    2578 flags.go:64] FLAG: --cluster-domain=""
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442192    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442195    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442198    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442202    2578 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442206    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442209    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442212    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442216    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442221    2578 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442225    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 18:00:12.588158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442231    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442237    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442249    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442254    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442257    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442260    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442264    2578 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442269    2578 flags.go:64] FLAG: --enable-server="true"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442273    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442277    2578 flags.go:64] FLAG: --event-burst="100"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442281    2578 flags.go:64] FLAG: --event-qps="50"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442284    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442287    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442290    2578 flags.go:64] FLAG: --eviction-hard=""
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442294    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442300    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442303    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442306    2578 flags.go:64] FLAG: --eviction-soft=""
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442309    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442314    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442317    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442320    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442324    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442329    2578 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442332    2578 flags.go:64] FLAG: --feature-gates=""
Jul 21 18:00:12.593317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442336    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442339    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442342    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442346    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442349    2578 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442352    2578 flags.go:64] FLAG: --help="false"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442355    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-140-73.ec2.internal"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442361    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442365    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442368    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442371    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442375    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442378    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442381    2578 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442384    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442388    2578 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442396    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442402    2578 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442406    2578 flags.go:64] FLAG: --kube-reserved=""
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442410    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442413    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442416    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442419    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442422    2578 flags.go:64] FLAG: --lock-file=""
Jul 21 18:00:12.594190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442427    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442430    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442436    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442441    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442444    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442447    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442450    2578 flags.go:64] FLAG: --logging-format="text"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442453    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442456    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442461    2578 flags.go:64] FLAG: --manifest-url=""
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442464    2578 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442469    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442472    2578 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442476    2578 flags.go:64] FLAG: --max-pods="110"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442480    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442483    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442486    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442492    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442495    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442498    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442501    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442509    2578 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442512    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442516    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 18:00:12.595047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442522    2578 flags.go:64] FLAG: --pod-cidr=""
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442525    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442530    2578 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442533    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442537    2578 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442540    2578 flags.go:64] FLAG: --port="10250"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442543    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442546    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0511b6d18a0890a33"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442550    2578 flags.go:64] FLAG: --qos-reserved=""
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442558    2578 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442569    2578 flags.go:64] FLAG: --register-node="true"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442575    2578 flags.go:64] FLAG: --register-with-taints=""
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442584    2578 flags.go:64] FLAG: --registry-burst="10"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442598    2578 flags.go:64] FLAG: --registry-qps="5"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442602    2578 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442605    2578 flags.go:64] FLAG: --reserved-memory=""
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442609    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442615    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442618    2578 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442621    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442624    2578 flags.go:64] FLAG: --runonce="false"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442628    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442631    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442634    2578 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442638    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 18:00:12.599351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442643    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442647    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442650    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442653    2578 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442656    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442659    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442664    2578 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442667    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442670    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442675    2578 flags.go:64] FLAG: --system-cgroups=""
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442678    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442684    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442687    2578 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442691    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442695    2578 flags.go:64] FLAG: --tls-min-version=""
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442698    2578 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442704    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442707    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442711    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442716    2578 flags.go:64] FLAG: --v="2"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442722    2578 flags.go:64] FLAG: --version="false"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442734    2578 flags.go:64] FLAG: --vmodule=""
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442740    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.442743    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442958    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 18:00:12.600217 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442963    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442970    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442973    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442976    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442978    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442981    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442984    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442986    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442989    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442992    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442995    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.442998    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443001    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443005    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443011    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443014    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443017    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443019    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443022    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443024    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 18:00:12.601045 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443027    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443030    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443032    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443035    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443039    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443046    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443061    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443065    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443069    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443072    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443075    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443078    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443081    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443084    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443087    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443089    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443092    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443095    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443101    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 18:00:12.603220 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443103    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443106    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443109    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443112    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443116    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443118    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443122    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443125    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443128    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443131    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443133    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443136    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443141    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443143    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443146    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443149    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443151    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443154    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443159    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 18:00:12.604052 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443163    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443166    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443169    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443172    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443175    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443181    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443184    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443187    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443190    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443193    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443195    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443198    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443201    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443204    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443208    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443213    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443223    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443229    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443234    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443238    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 18:00:12.709325 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443241    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443244    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443246    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443249    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443252    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443254    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443257    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443260    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443262    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443265    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443270    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443274    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443277    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443280    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443282    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443285    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443288    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443292    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443295    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 18:00:12.710166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.443298    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 18:00:12.713619 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.444945    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 18:00:12.902837 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.902805    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 18:00:12.902837 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.902834    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.902850    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.902860    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902926    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902932    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902936    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902939    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902942    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902945    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902948    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902951    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902954    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902957    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902959    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902962    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902965    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902968    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902970    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902973    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 18:00:12.902968 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902976    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902979    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902982    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902985    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902987    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902990    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902993    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902995    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.902998    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903001    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903003    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903006    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903008    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903011    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903013    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903017    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903019    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903022    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903024    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903027    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 18:00:12.903418 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903030    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903033    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903035    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903038    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903040    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903043    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903045    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903049    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903051    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903054    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903056    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903059    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903061    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903064    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903068    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903072    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903075    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903078    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903081    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 18:00:12.903905 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903083    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903086    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903089    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903092    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903094    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903097    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903099    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903102    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903104    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903107    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903110    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903112    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903115    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903119    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903123    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903126    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903128    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903131    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903134    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903137    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 18:00:12.904352 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903139    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903142    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903144    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903147    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903149    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903152    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903156    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903160    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903164    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903167    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903169    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903173    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903175    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903178    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903181    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903184    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903186    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903188    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903191    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 18:00:12.904847 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903193    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903196    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903198    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903201    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903203    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.903208    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903330    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903334    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903337    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903340    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903343    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903346    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903349    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903352    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903355    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 18:00:12.905298 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903357    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903360    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903363    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903366    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903368    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903371    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903374    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903377    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903380    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903383    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903385    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903389    2578 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903391    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903394    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903397    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903400    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903402    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903405    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903407    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903410    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 18:00:12.905680 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903412    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903415    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903417    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903420    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903422    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903425    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903428    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903430    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903433    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903435    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903438    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903441    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903443    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903446    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903448    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903450    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903453    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903456    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903460    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903463    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 18:00:12.906166 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903465    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903468    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903471    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903473    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903476    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903478    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903482    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903484    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903487    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903490    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903493    2578 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903496    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903500    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903504    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903507    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903510    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903512    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903515    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903518    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903520    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 18:00:12.906749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903523    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903526    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903528    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903531    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903533    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903536    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903538    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903541    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903543    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903546    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903549    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903552    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903555    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903557    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903560    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903562    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903565    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903569    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903572    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 18:00:12.907215 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903575    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903579    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903582    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903585    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903600    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903603    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903606    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903609    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903612    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903615    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:12.903618    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.903623    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.903787    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.906095    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 18:00:12.907684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.907357    2578 server.go:1019] "Starting client certificate rotation"
Jul 21 18:00:12.908026 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.907463    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 18:00:12.908026 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.907496    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 18:00:12.909608 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.909582    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 18:00:12.912426 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.912393    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 18:00:12.926123 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.926105    2578 log.go:25] "Validated CRI v1 runtime API"
Jul 21 18:00:12.927968 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.927949    2578 log.go:25] "Validated CRI v1 image API"
Jul 21 18:00:12.928751 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.928737    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 18:00:12.931860 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.931841    2578 fs.go:135] Filesystem UUIDs: map[6b0380b8-bbcd-4569-ab54-85a05cb562c8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ea42cdf6-cb40-4cd5-b7a3-01da486b876f:/dev/nvme0n1p4]
Jul 21 18:00:12.931929 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.931860    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 18:00:12.935359 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.935344    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 18:00:12.936850 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.936725    2578 manager.go:216] Machine: {Timestamp:2026-07-21 18:00:12.935403551 +0000 UTC m=+1.193846722 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099617 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21db32680c02fa5c572af48f46a9ee SystemUUID:ec21db32-680c-02fa-5c57-2af48f46a9ee BootID:37d61168-ba28-4ce6-b9fe-7940ed417c85 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4f:50:8e:ee:65 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4f:50:8e:ee:65 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:9c:62:46:9b:5d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 18:00:12.936850 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.936837    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 18:00:12.936974 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.936956    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 18:00:12.940671 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.940468    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 18:00:12.940810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.940675    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-73.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 18:00:12.941678 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.941668    2578 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 18:00:12.941716 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.941680    2578 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 18:00:12.941716 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.941687    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 18:00:12.942317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.942306    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 18:00:12.943333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.943323    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 18:00:12.945660 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.945648    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 18:00:12.945778 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.945769    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 18:00:12.945806 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.945801    2578 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 18:00:12.945841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.945812    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 18:00:12.945841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.945826    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 18:00:12.945841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.945835    2578 kubelet.go:398] "Adding apiserver pod source"
Jul 21 18:00:12.945964 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.945844    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 18:00:12.946274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.946262    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 18:00:12.946320 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.946281    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 18:00:12.948482 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.948466    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 18:00:12.950198 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.950180    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 18:00:12.950290 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.950219    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 18:00:12.951335 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951321    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951341    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951350    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951358    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951370    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951379    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951390    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951398    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951408    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951416    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951428    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 18:00:12.951464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.951436    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 18:00:12.952404 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.952392    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 18:00:12.952459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.952407    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 18:00:12.955772 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.955756    2578 server.go:1268] "Started kubelet"
Jul 21 18:00:12.956193 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.955817    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 18:00:12.956193 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.955862    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 18:00:12.956193 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.955946    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 18:00:12.956548 ip-10-0-140-73 systemd[1]: Started Kubernetes Kubelet.
Jul 21 18:00:12.957646 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.957548    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 18:00:12.958049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.958034    2578 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 18:00:12.958919 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.958893    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-73.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 18:00:12.959541 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.959482    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 18:00:12.959643 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.959539    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-73.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 18:00:12.962098 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.962080    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 18:00:12.962400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.962384    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 18:00:12.962467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.962400    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 18:00:12.962519 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.962461    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:12.962810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.962788    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 18:00:12.962810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.962808    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 18:00:12.962953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.962830    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 18:00:12.962953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.962918    2578 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 18:00:12.962953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.962938    2578 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 18:00:12.963477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.963452    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 18:00:12.963477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.963470    2578 factory.go:55] Registering systemd factory
Jul 21 18:00:12.963477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.963481    2578 factory.go:223] Registration of the systemd container factory successfully
Jul 21 18:00:12.963734 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.963717    2578 factory.go:153] Registering CRI-O factory
Jul 21 18:00:12.963734 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.963733    2578 factory.go:223] Registration of the crio container factory successfully
Jul 21 18:00:12.963842 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.963755    2578 factory.go:103] Registering Raw factory
Jul 21 18:00:12.963842 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.963772    2578 manager.go:1196] Started watching for new ooms in manager
Jul 21 18:00:12.964258 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.964240    2578 manager.go:349] Starting recovery of all containers
Jul 21 18:00:12.964374 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.964354    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 21 18:00:12.964490 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.964459    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-73.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 21 18:00:12.967882 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.964259    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18c45fea76325c97  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-07-21 18:00:12.955557015 +0000 UTC m=+1.214000186,LastTimestamp:2026-07-21 18:00:12.955557015 +0000 UTC m=+1.214000186,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}"
Jul 21 18:00:12.968322 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.968292    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6nmf9"
Jul 21 18:00:12.971980 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.971115    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6nmf9"
Jul 21 18:00:12.972950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.972921    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 18:00:12.979004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.978988    2578 manager.go:354] Recovery completed
Jul 21 18:00:12.983764 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.983752    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 18:00:12.985511 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.985492    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 18:00:12.985586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.985531    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 18:00:12.985586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.985548    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID"
Jul 21 18:00:12.986212 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.986198    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 18:00:12.986212 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.986210    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 18:00:12.986325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.986229    2578 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 18:00:12.986325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.986212    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 18:00:12.987009 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.986994    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 18:00:12.987077 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.987023    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 18:00:12.987077 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.987038    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID"
Jul 21 18:00:12.987336 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.987323    2578 policy_none.go:49] "None policy: Start"
Jul 21 18:00:12.987386 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.987339    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 18:00:12.987386 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.987350    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 18:00:12.988151 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:12.988137    2578 policy_none.go:47] "Start"
Jul 21 18:00:12.988151 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:12.988084    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-73.ec2.internal.18c45fea77fb721b  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-73.ec2.internal,UID:ip-10-0-140-73.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-140-73.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-140-73.ec2.internal,},FirstTimestamp:2026-07-21 18:00:12.985512475 +0000 UTC m=+1.243955649,LastTimestamp:2026-07-21 18:00:12.985512475 +0000 UTC m=+1.243955649,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-73.ec2.internal,}"
Jul 21 18:00:13.022170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.022147    2578 manager.go:337] "Starting Device Plugin manager"
Jul 21 18:00:13.022290 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.022185    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 18:00:13.022290 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.022195    2578 server.go:87] "Starting device plugin registration server"
Jul 21 18:00:13.022470 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.022459    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 18:00:13.022514 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.022471    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 18:00:13.022604 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.022574    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 18:00:13.022926 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.022666    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 18:00:13.022926 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.022678    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 18:00:13.024266 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.024242    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 18:00:13.024337 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.024304    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.095547 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.095507    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 18:00:13.095547 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.095555    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 18:00:13.096656 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.095577    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 18:00:13.096656 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.095625    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 18:00:13.098049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.098018    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 18:00:13.123150 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.123128    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 18:00:13.124583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.124564    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 18:00:13.124690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.124614    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 18:00:13.124690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.124628    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID"
Jul 21 18:00:13.124690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.124657    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.131343 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.131330    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.131398 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.131351    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-73.ec2.internal\": node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.144165 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.144145    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.196741 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.196699    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal"]
Jul 21 18:00:13.196875 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.196794    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 18:00:13.197866 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.197852    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 18:00:13.197929 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.197882    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 18:00:13.197929 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.197896    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID"
Jul 21 18:00:13.199022 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199009    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 18:00:13.199143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199126    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.199198 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199173    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 18:00:13.199737 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199723    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 18:00:13.199790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199740    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 18:00:13.199790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199750    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 18:00:13.199790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199761    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 18:00:13.199790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199765    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID"
Jul 21 18:00:13.199790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.199771    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID"
Jul 21 18:00:13.200774 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.200759    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.200818 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.200790    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 18:00:13.201459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.201443    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 18:00:13.201533 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.201468    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 18:00:13.201533 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.201483    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeHasSufficientPID"
Jul 21 18:00:13.220237 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.220182    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-73.ec2.internal\" not found" node="ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.223939 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.223923    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-73.ec2.internal\" not found" node="ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.245213 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.245188    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.264586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.264554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fba785e28d682960c2716b1944cd9f41-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"fba785e28d682960c2716b1944cd9f41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.264727 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.264603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fba785e28d682960c2716b1944cd9f41-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"fba785e28d682960c2716b1944cd9f41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.264727 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.264633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd118746fb461e4f1c5de51b2e36feed-config\") pod \"kube-apiserver-proxy-ip-10-0-140-73.ec2.internal\" (UID: \"fd118746fb461e4f1c5de51b2e36feed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.345659 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.345628    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.365025 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.365000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fba785e28d682960c2716b1944cd9f41-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"fba785e28d682960c2716b1944cd9f41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.365138 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.365036    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fba785e28d682960c2716b1944cd9f41-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"fba785e28d682960c2716b1944cd9f41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.365138 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.365063    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fba785e28d682960c2716b1944cd9f41-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"fba785e28d682960c2716b1944cd9f41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.365138 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.365061    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd118746fb461e4f1c5de51b2e36feed-config\") pod \"kube-apiserver-proxy-ip-10-0-140-73.ec2.internal\" (UID: \"fd118746fb461e4f1c5de51b2e36feed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.365138 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.365094    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fba785e28d682960c2716b1944cd9f41-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal\" (UID: \"fba785e28d682960c2716b1944cd9f41\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.365138 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.365097    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/fd118746fb461e4f1c5de51b2e36feed-config\") pod \"kube-apiserver-proxy-ip-10-0-140-73.ec2.internal\" (UID: \"fd118746fb461e4f1c5de51b2e36feed\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.446326 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.446283    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.522935 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.522869    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.526800 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.526779    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:13.546705 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.546522    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.647097 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.647054    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.747568 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.747541    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.848295 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.848212    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.906666 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.906643    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 18:00:13.906841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.906816    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 18:00:13.949107 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:13.949081    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:13.976622 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.976570    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 17:55:12 +0000 UTC" deadline="2028-05-02 21:47:56.861164434 +0000 UTC"
Jul 21 18:00:13.976622 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:13.976624    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15627h47m42.884545844s"
Jul 21 18:00:14.049657 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:14.049632    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-73.ec2.internal\" not found"
Jul 21 18:00:14.093189 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.093167    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 18:00:14.161672 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:14.161639    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfba785e28d682960c2716b1944cd9f41.slice/crio-88b3e3f7acbd07e316f9f613ed31c68044f75c54a455d0a8d4e7ecddd502a3a0 WatchSource:0}: Error finding container 88b3e3f7acbd07e316f9f613ed31c68044f75c54a455d0a8d4e7ecddd502a3a0: Status 404 returned error can't find the container with id 88b3e3f7acbd07e316f9f613ed31c68044f75c54a455d0a8d4e7ecddd502a3a0
Jul 21 18:00:14.161874 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:14.161855    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd118746fb461e4f1c5de51b2e36feed.slice/crio-b7752ea96ed6595c799988aedbf27b843312968aa5038fdb7f1ad9d85d2b3278 WatchSource:0}: Error finding container b7752ea96ed6595c799988aedbf27b843312968aa5038fdb7f1ad9d85d2b3278: Status 404 returned error can't find the container with id b7752ea96ed6595c799988aedbf27b843312968aa5038fdb7f1ad9d85d2b3278
Jul 21 18:00:14.162944 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.162929    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:14.167380 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.167365    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 18:00:14.171209 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.171189    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 18:00:14.173087 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.173071    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal"
Jul 21 18:00:14.181688 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.181668    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 18:00:14.182826 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.182813    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 18:00:14.434330 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.434246    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 18:00:14.789451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.789378    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 18:00:14.947341 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.947304    2578 apiserver.go:52] "Watching apiserver"
Jul 21 18:00:14.956180 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.956157    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 18:00:14.957478 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.957455    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zqmhv","kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw","openshift-multus/multus-cvv4k","openshift-multus/network-metrics-daemon-pkx8c","openshift-network-diagnostics/network-check-target-rvdsz","openshift-network-operator/iptables-alerter-hkl9f","openshift-ovn-kubernetes/ovnkube-node-7dgs2","kube-system/konnectivity-agent-vss4j","openshift-cluster-node-tuning-operator/tuned-2mq2m","openshift-dns/node-resolver-2q8df","openshift-image-registry/node-ca-xpnv6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal","openshift-multus/multus-additional-cni-plugins-p9m59"]
Jul 21 18:00:14.959651 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.959635    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:14.960938 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.960911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:14.962241 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.962218    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.962856 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.962838    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bscjt\""
Jul 21 18:00:14.962942 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.962863    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 18:00:14.962942 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.962888    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 18:00:14.963325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.963308    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.963509 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.963451    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 18:00:14.963630 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.963608    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-7dnrj\""
Jul 21 18:00:14.964019 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.964002    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 18:00:14.964680 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.964649    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:14.964779 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:14.964719    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:14.964869 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.964854    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 18:00:14.964945 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.964877    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 18:00:14.965421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.965383    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 18:00:14.965775 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.965662    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:14.965775 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:14.965728    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:14.966287 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.966266    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 18:00:14.966725 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.966704    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 18:00:14.966830 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.966709    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 18:00:14.967055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.967020    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:14.968291 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.968267    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bqkb7\""
Jul 21 18:00:14.968720 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.968702    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r4mhs\""
Jul 21 18:00:14.968802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.968790    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 18:00:14.968918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.968899    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 18:00:14.969424 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.969405    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 18:00:14.970000 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.969982    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:14.971912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.971894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:14.973070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-netns\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973084    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-cni-bin\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973104    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-conf-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973125    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2gtd\" (UniqueName: \"kubernetes.io/projected/548f9041-5dfc-4edb-86e3-0e0f276ac154-kube-api-access-d2gtd\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:14.973166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973155    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-8n87w\""
Jul 21 18:00:14.973378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973182    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdfba343-022f-434b-b69a-64677d0384c5-hosts-file\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:14.973378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.973378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-sys-fs\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.973378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973329    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 18:00:14.973378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxp4v\" (UniqueName: \"kubernetes.io/projected/bb360206-b696-49f2-a38c-34523120782d-kube-api-access-nxp4v\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.973378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973373    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973385    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-cnibin\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973423    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-os-release\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973452    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-k8s-cni-cncf-io\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973480    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-cni-multus\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973502    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/548f9041-5dfc-4edb-86e3-0e0f276ac154-host-slash\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973560    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973620    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-device-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-hostroot\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.973701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-multus-certs\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-etc-kubernetes\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973758    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-socket-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973782    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-registration-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-kubelet\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973835    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de361996-b492-47df-8f47-24e14804473e-multus-daemon-config\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973847    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973857    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-system-cni-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973883    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-cni-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/548f9041-5dfc-4edb-86e3-0e0f276ac154-iptables-alerter-script\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdfba343-022f-434b-b69a-64677d0384c5-tmp-dir\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973968    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de361996-b492-47df-8f47-24e14804473e-cni-binary-copy\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.973992    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-socket-dir-parent\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.974015    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhhr4\" (UniqueName: \"kubernetes.io/projected/de361996-b492-47df-8f47-24e14804473e-kube-api-access-qhhr4\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:14.974117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.974049    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj5kg\" (UniqueName: \"kubernetes.io/projected/fdfba343-022f-434b-b69a-64677d0384c5-kube-api-access-fj5kg\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:14.974794 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.974477    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 18:00:14.974794 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.974497    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 18:00:14.974794 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.974763    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sh8dg\""
Jul 21 18:00:14.974933 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.974802    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-q896n\""
Jul 21 18:00:14.975015 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.975000    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 18:00:14.975230 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.975125    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:14.975230 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:14.975220    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:14.975739 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.975720    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:14.977418 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.977401    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-tkz44\""
Jul 21 18:00:14.977418 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.977413    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 18:00:14.977555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.977508    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 18:00:14.977555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.977537    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 18:00:14.978951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.978846    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 18:00:14.978951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.978869    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 18:00:14.978951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.978895    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 18:00:14.978951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.978931    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 18:00:14.979197 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.979152    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 18:00:14.979197 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.979178    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 18:00:14.979618 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.979542    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 18:00:14.979618 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.979546    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zmp9x\""
Jul 21 18:00:14.979618 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:14.979606    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 18:00:15.002305 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.002283    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-9f27k"
Jul 21 18:00:15.009156 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.009132    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-9f27k"
Jul 21 18:00:15.064255 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.064194    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 18:00:15.074903 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.074875    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdfba343-022f-434b-b69a-64677d0384c5-tmp-dir\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:15.075068 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.074920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-systemd\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.075068 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.074949    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.075207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075080    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdcq6\" (UniqueName: \"kubernetes.io/projected/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-kube-api-access-kdcq6\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.075207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de361996-b492-47df-8f47-24e14804473e-cni-binary-copy\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.075207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075139    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-socket-dir-parent\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.075207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075155    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-sys\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.075207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-var-lib-kubelet\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.075207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075186    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.075207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj5kg\" (UniqueName: \"kubernetes.io/projected/fdfba343-022f-434b-b69a-64677d0384c5-kube-api-access-fj5kg\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-modprobe-d\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075232    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysctl-d\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075245    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-systemd\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d433d8-39ce-40a0-a984-13aeea485f20-etc-tuned\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075273    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kl4l\" (UniqueName: \"kubernetes.io/projected/0710eca1-fe6f-4b08-a308-1dbe4e70619e-kube-api-access-4kl4l\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075288    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-cni-netd\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovnkube-script-lib\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075349    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-cni-bin\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-conf-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075395    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2gtd\" (UniqueName: \"kubernetes.io/projected/548f9041-5dfc-4edb-86e3-0e0f276ac154-kube-api-access-d2gtd\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075422    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdfba343-022f-434b-b69a-64677d0384c5-hosts-file\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:15.075573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e588e98b-3542-4fdd-afa9-fb8b1e3d787e-agent-certs\") pod \"konnectivity-agent-vss4j\" (UID: \"e588e98b-3542-4fdd-afa9-fb8b1e3d787e\") " pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:15.076143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075882    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.076143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-run-netns\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.076143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/de361996-b492-47df-8f47-24e14804473e-cni-binary-copy\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.076364 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-cni-bin\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.076429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076359    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-socket-dir-parent\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.076429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.075307    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdfba343-022f-434b-b69a-64677d0384c5-tmp-dir\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:15.076623 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076453    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d858ae26-2013-4bb3-be07-621fa975f0f1-serviceca\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.076623 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/fdfba343-022f-434b-b69a-64677d0384c5-hosts-file\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:15.076763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076636    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-cnibin\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.076763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076673    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-os-release\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.076898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-k8s-cni-cncf-io\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.076898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076849    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-cnibin\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.077015 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/548f9041-5dfc-4edb-86e3-0e0f276ac154-host-slash\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:15.077015 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.076980    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysconfig\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.077015 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077011    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d433d8-39ce-40a0-a984-13aeea485f20-tmp\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.077330 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:15.077451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077348    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-node-log\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.077451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077382    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gx58\" (UniqueName: \"kubernetes.io/projected/d858ae26-2013-4bb3-be07-621fa975f0f1-kube-api-access-9gx58\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.077451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-multus-certs\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.077614 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077523    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-kubernetes\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.077614 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysctl-conf\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.077614 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-lib-modules\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077871    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/548f9041-5dfc-4edb-86e3-0e0f276ac154-host-slash\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-ovn\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077927    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-conf-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077944    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-multus-certs\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-k8s-cni-cncf-io\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-os-release\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.077994    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-etc-kubernetes\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078033    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-registration-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078075    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-registration-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-kubelet\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078088    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-etc-kubernetes\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-kubelet\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078131    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nxtb\" (UniqueName: \"kubernetes.io/projected/0ef3f2e2-9940-4e62-a294-673fb6176efc-kube-api-access-8nxtb\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078189    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-var-lib-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078218    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-etc-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-system-cni-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.079329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-cni-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e588e98b-3542-4fdd-afa9-fb8b1e3d787e-konnectivity-ca\") pod \"konnectivity-agent-vss4j\" (UID: \"e588e98b-3542-4fdd-afa9-fb8b1e3d787e\") " pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cnibin\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078534    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-multus-cni-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078572    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhhr4\" (UniqueName: \"kubernetes.io/projected/de361996-b492-47df-8f47-24e14804473e-kube-api-access-qhhr4\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-run\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqv8r\" (UniqueName: \"kubernetes.io/projected/37d433d8-39ce-40a0-a984-13aeea485f20-kube-api-access-lqv8r\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078681    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-cni-bin\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078755    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df1e655a-c0cc-4d13-8716-6a0fe968b90d-kubelet-config\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df1e655a-c0cc-4d13-8716-6a0fe968b90d-dbus\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-os-release\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078839    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-slash\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-env-overrides\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d858ae26-2013-4bb3-be07-621fa975f0f1-host\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.078925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-netns\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-host\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079032    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cni-binary-copy\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.080191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079056    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-kubelet\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-systemd-units\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079147    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-log-socket\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079178    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079209    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-sys-fs\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079234    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxp4v\" (UniqueName: \"kubernetes.io/projected/bb360206-b696-49f2-a38c-34523120782d-kube-api-access-nxp4v\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079268    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-run-netns\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-cni-multus\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.079976    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-system-cni-dir\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080027    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080072    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080115    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-host-var-lib-cni-multus\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080206    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-sys-fs\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080229    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-system-cni-dir\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080313    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-etc-selinux\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080375    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-device-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080406    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-hostroot\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/de361996-b492-47df-8f47-24e14804473e-hostroot\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080464    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080489    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-device-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080528    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovnkube-config\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-socket-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.080998    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bb360206-b696-49f2-a38c-34523120782d-socket-dir\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.081049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de361996-b492-47df-8f47-24e14804473e-multus-daemon-config\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.081116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.081185    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.081219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/548f9041-5dfc-4edb-86e3-0e0f276ac154-iptables-alerter-script\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.081916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/de361996-b492-47df-8f47-24e14804473e-multus-daemon-config\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.081946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.081930    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/548f9041-5dfc-4edb-86e3-0e0f276ac154-iptables-alerter-script\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:15.082677 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.082501    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 18:00:15.086493 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.086470    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj5kg\" (UniqueName: \"kubernetes.io/projected/fdfba343-022f-434b-b69a-64677d0384c5-kube-api-access-fj5kg\") pod \"node-resolver-2q8df\" (UID: \"fdfba343-022f-434b-b69a-64677d0384c5\") " pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:15.086493 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.086486    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2gtd\" (UniqueName: \"kubernetes.io/projected/548f9041-5dfc-4edb-86e3-0e0f276ac154-kube-api-access-d2gtd\") pod \"iptables-alerter-hkl9f\" (UID: \"548f9041-5dfc-4edb-86e3-0e0f276ac154\") " pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:15.088927 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.088884    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxp4v\" (UniqueName: \"kubernetes.io/projected/bb360206-b696-49f2-a38c-34523120782d-kube-api-access-nxp4v\") pod \"aws-ebs-csi-driver-node-lckjw\" (UID: \"bb360206-b696-49f2-a38c-34523120782d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.089012 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.089003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhhr4\" (UniqueName: \"kubernetes.io/projected/de361996-b492-47df-8f47-24e14804473e-kube-api-access-qhhr4\") pod \"multus-cvv4k\" (UID: \"de361996-b492-47df-8f47-24e14804473e\") " pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.101178 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.101122    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" event={"ID":"fd118746fb461e4f1c5de51b2e36feed","Type":"ContainerStarted","Data":"b7752ea96ed6595c799988aedbf27b843312968aa5038fdb7f1ad9d85d2b3278"}
Jul 21 18:00:15.102440 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.102414    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" event={"ID":"fba785e28d682960c2716b1944cd9f41","Type":"ContainerStarted","Data":"88b3e3f7acbd07e316f9f613ed31c68044f75c54a455d0a8d4e7ecddd502a3a0"}
Jul 21 18:00:15.181815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-kubernetes\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.181815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181811    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysctl-conf\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181836    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-lib-modules\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181859    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-ovn\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxtb\" (UniqueName: \"kubernetes.io/projected/0ef3f2e2-9940-4e62-a294-673fb6176efc-kube-api-access-8nxtb\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-kubernetes\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181901    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-var-lib-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181950    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-var-lib-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysctl-conf\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.181989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-etc-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182003    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-ovn\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e588e98b-3542-4fdd-afa9-fb8b1e3d787e-konnectivity-ca\") pod \"konnectivity-agent-vss4j\" (UID: \"e588e98b-3542-4fdd-afa9-fb8b1e3d787e\") " pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-etc-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cnibin\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182070    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-lib-modules\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182074    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-run\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182119    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cnibin\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182124    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-run\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182135    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqv8r\" (UniqueName: \"kubernetes.io/projected/37d433d8-39ce-40a0-a984-13aeea485f20-kube-api-access-lqv8r\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182213    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-cni-bin\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182239    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df1e655a-c0cc-4d13-8716-6a0fe968b90d-kubelet-config\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df1e655a-c0cc-4d13-8716-6a0fe968b90d-dbus\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182284    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-os-release\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182305    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-slash\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-env-overrides\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182352    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d858ae26-2013-4bb3-be07-621fa975f0f1-host\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182376    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-host\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182400    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cni-binary-copy\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182423    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182436    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-os-release\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.182699 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182448    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-kubelet\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-systemd-units\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-log-socket\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182503    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/df1e655a-c0cc-4d13-8716-6a0fe968b90d-kubelet-config\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182557    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/df1e655a-c0cc-4d13-8716-6a0fe968b90d-dbus\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182558    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-system-cni-dir\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182615    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-system-cni-dir\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182638    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e588e98b-3542-4fdd-afa9-fb8b1e3d787e-konnectivity-ca\") pod \"konnectivity-agent-vss4j\" (UID: \"e588e98b-3542-4fdd-afa9-fb8b1e3d787e\") " pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-slash\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-kubelet\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182703    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovnkube-config\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-systemd-units\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182733    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-log-socket\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.183485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-host\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182841    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-openvswitch\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d858ae26-2013-4bb3-be07-621fa975f0f1-host\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182697    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-cni-bin\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182923    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-systemd\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182930    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182949    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.182982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdcq6\" (UniqueName: \"kubernetes.io/projected/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-kube-api-access-kdcq6\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.183006    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-sys\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.183071    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret podName:df1e655a-c0cc-4d13-8716-6a0fe968b90d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:15.683053992 +0000 UTC m=+3.941497172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret") pod "global-pull-secret-syncer-zqmhv" (UID: "df1e655a-c0cc-4d13-8716-6a0fe968b90d") : object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183007    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-sys\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183095    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-var-lib-kubelet\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183110    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-env-overrides\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.183132    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:15.184340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-modprobe-d\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.183176    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs podName:0ef3f2e2-9940-4e62-a294-673fb6176efc nodeName:}" failed. No retries permitted until 2026-07-21 18:00:15.683161552 +0000 UTC m=+3.941604723 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs") pod "network-metrics-daemon-pkx8c" (UID: "0ef3f2e2-9940-4e62-a294-673fb6176efc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysctl-d\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183241    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-modprobe-d\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-systemd\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183279    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d433d8-39ce-40a0-a984-13aeea485f20-etc-tuned\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kl4l\" (UniqueName: \"kubernetes.io/projected/0710eca1-fe6f-4b08-a308-1dbe4e70619e-kube-api-access-4kl4l\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183323    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cni-binary-copy\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183327    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-cni-netd\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183343    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovnkube-script-lib\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183385    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0710eca1-fe6f-4b08-a308-1dbe4e70619e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e588e98b-3542-4fdd-afa9-fb8b1e3d787e-agent-certs\") pod \"konnectivity-agent-vss4j\" (UID: \"e588e98b-3542-4fdd-afa9-fb8b1e3d787e\") " pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183428    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183438    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysctl-d\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183458    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-run-netns\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.185188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183484    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d858ae26-2013-4bb3-be07-621fa975f0f1-serviceca\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183514    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysconfig\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-run-ovn-kubernetes\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d433d8-39ce-40a0-a984-13aeea485f20-tmp\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183614    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-node-log\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gx58\" (UniqueName: \"kubernetes.io/projected/d858ae26-2013-4bb3-be07-621fa975f0f1-kube-api-access-9gx58\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183769    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-sysconfig\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovnkube-config\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183895    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-cni-netd\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183909    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0710eca1-fe6f-4b08-a308-1dbe4e70619e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183963    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-var-lib-kubelet\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/37d433d8-39ce-40a0-a984-13aeea485f20-etc-systemd\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.184013    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-node-log\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183847    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovnkube-script-lib\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.184084    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-host-run-netns\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.183485    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-run-systemd\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.186083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.184402    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d858ae26-2013-4bb3-be07-621fa975f0f1-serviceca\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.186936 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.186701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-ovn-node-metrics-cert\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.187515 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.187494    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/37d433d8-39ce-40a0-a984-13aeea485f20-tmp\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.188320 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.188277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e588e98b-3542-4fdd-afa9-fb8b1e3d787e-agent-certs\") pod \"konnectivity-agent-vss4j\" (UID: \"e588e98b-3542-4fdd-afa9-fb8b1e3d787e\") " pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:15.188429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.188406    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/37d433d8-39ce-40a0-a984-13aeea485f20-etc-tuned\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.193395 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.191709    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 18:00:15.193395 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.191733    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 18:00:15.193395 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.191749    2578 projected.go:196] Error preparing data for projected volume kube-api-access-2ddn6 for pod openshift-network-diagnostics/network-check-target-rvdsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:15.193395 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.191815    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6 podName:b34e9272-5563-42b1-acce-2e650f738f69 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:15.691798543 +0000 UTC m=+3.950241704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2ddn6" (UniqueName: "kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6") pod "network-check-target-rvdsz" (UID: "b34e9272-5563-42b1-acce-2e650f738f69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:15.193395 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.192736    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nxtb\" (UniqueName: \"kubernetes.io/projected/0ef3f2e2-9940-4e62-a294-673fb6176efc-kube-api-access-8nxtb\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:15.194564 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.194540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqv8r\" (UniqueName: \"kubernetes.io/projected/37d433d8-39ce-40a0-a984-13aeea485f20-kube-api-access-lqv8r\") pod \"tuned-2mq2m\" (UID: \"37d433d8-39ce-40a0-a984-13aeea485f20\") " pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.195750 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.195729    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdcq6\" (UniqueName: \"kubernetes.io/projected/f3be15ef-cbc8-43ee-862b-c8b341d07fb4-kube-api-access-kdcq6\") pod \"ovnkube-node-7dgs2\" (UID: \"f3be15ef-cbc8-43ee-862b-c8b341d07fb4\") " pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.196715 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.196655    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gx58\" (UniqueName: \"kubernetes.io/projected/d858ae26-2013-4bb3-be07-621fa975f0f1-kube-api-access-9gx58\") pod \"node-ca-xpnv6\" (UID: \"d858ae26-2013-4bb3-be07-621fa975f0f1\") " pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.196715 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.196654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kl4l\" (UniqueName: \"kubernetes.io/projected/0710eca1-fe6f-4b08-a308-1dbe4e70619e-kube-api-access-4kl4l\") pod \"multus-additional-cni-plugins-p9m59\" (UID: \"0710eca1-fe6f-4b08-a308-1dbe4e70619e\") " pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.272137 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.272109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hkl9f"
Jul 21 18:00:15.280480 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.280457    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2q8df"
Jul 21 18:00:15.287134 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.287117    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw"
Jul 21 18:00:15.293362 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.293341    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cvv4k"
Jul 21 18:00:15.299174 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.299158    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:15.302961 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:15.302911    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb360206_b696_49f2_a38c_34523120782d.slice/crio-62974b3821fe464975bbbac47292716e66400c1f5d1e7eaf3ca81d1a6f43af72 WatchSource:0}: Error finding container 62974b3821fe464975bbbac47292716e66400c1f5d1e7eaf3ca81d1a6f43af72: Status 404 returned error can't find the container with id 62974b3821fe464975bbbac47292716e66400c1f5d1e7eaf3ca81d1a6f43af72
Jul 21 18:00:15.309348 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:15.305123    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde361996_b492_47df_8f47_24e14804473e.slice/crio-456c91fe4af92f46e957f4b87dc941cf6b96d02dadfa9abb2eeee36272e34afb WatchSource:0}: Error finding container 456c91fe4af92f46e957f4b87dc941cf6b96d02dadfa9abb2eeee36272e34afb: Status 404 returned error can't find the container with id 456c91fe4af92f46e957f4b87dc941cf6b96d02dadfa9abb2eeee36272e34afb
Jul 21 18:00:15.309348 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.306876    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-p9m59"
Jul 21 18:00:15.312650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.312630    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:15.318102 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:15.318078    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode588e98b_3542_4fdd_afa9_fb8b1e3d787e.slice/crio-bc0156369a9b3482e51a221078f78993348c2e347d254912bff535db60b9fffb WatchSource:0}: Error finding container bc0156369a9b3482e51a221078f78993348c2e347d254912bff535db60b9fffb: Status 404 returned error can't find the container with id bc0156369a9b3482e51a221078f78993348c2e347d254912bff535db60b9fffb
Jul 21 18:00:15.319139 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.319118    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2mq2m"
Jul 21 18:00:15.325297 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.325133    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xpnv6"
Jul 21 18:00:15.330053 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:15.330007    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0710eca1_fe6f_4b08_a308_1dbe4e70619e.slice/crio-8797e3029bbe122fe166cc50830e8ff2e59df99216e81b1305f0398c39124291 WatchSource:0}: Error finding container 8797e3029bbe122fe166cc50830e8ff2e59df99216e81b1305f0398c39124291: Status 404 returned error can't find the container with id 8797e3029bbe122fe166cc50830e8ff2e59df99216e81b1305f0398c39124291
Jul 21 18:00:15.334208 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:15.330912    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37d433d8_39ce_40a0_a984_13aeea485f20.slice/crio-69359d8fbe62f9dc8f6c6cbab3d5f45ff259ea9c406ac8cd333d4394eb6fc257 WatchSource:0}: Error finding container 69359d8fbe62f9dc8f6c6cbab3d5f45ff259ea9c406ac8cd333d4394eb6fc257: Status 404 returned error can't find the container with id 69359d8fbe62f9dc8f6c6cbab3d5f45ff259ea9c406ac8cd333d4394eb6fc257
Jul 21 18:00:15.334208 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:15.331926    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3be15ef_cbc8_43ee_862b_c8b341d07fb4.slice/crio-429a63d9c2626d1901ecb90a55e85b1c1ce96a3a1ddf0ec7f08786ccc96a0340 WatchSource:0}: Error finding container 429a63d9c2626d1901ecb90a55e85b1c1ce96a3a1ddf0ec7f08786ccc96a0340: Status 404 returned error can't find the container with id 429a63d9c2626d1901ecb90a55e85b1c1ce96a3a1ddf0ec7f08786ccc96a0340
Jul 21 18:00:15.343656 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:15.343622    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd858ae26_2013_4bb3_be07_621fa975f0f1.slice/crio-02a245a16840aa1242e2332169cc69c063a01d023a770159e0af92e18ae050a2 WatchSource:0}: Error finding container 02a245a16840aa1242e2332169cc69c063a01d023a770159e0af92e18ae050a2: Status 404 returned error can't find the container with id 02a245a16840aa1242e2332169cc69c063a01d023a770159e0af92e18ae050a2
Jul 21 18:00:15.687014 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.686982    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:15.687014 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.687018    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:15.687263 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.687143    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:15.687263 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.687166    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:15.687263 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.687208    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret podName:df1e655a-c0cc-4d13-8716-6a0fe968b90d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:16.687188075 +0000 UTC m=+4.945631243 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret") pod "global-pull-secret-syncer-zqmhv" (UID: "df1e655a-c0cc-4d13-8716-6a0fe968b90d") : object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:15.687263 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.687223    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs podName:0ef3f2e2-9940-4e62-a294-673fb6176efc nodeName:}" failed. No retries permitted until 2026-07-21 18:00:16.687216989 +0000 UTC m=+4.945660151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs") pod "network-metrics-daemon-pkx8c" (UID: "0ef3f2e2-9940-4e62-a294-673fb6176efc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:15.787877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:15.787842    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:15.788040 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.787995    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 18:00:15.788040 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.788015    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 18:00:15.788210 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.788041    2578 projected.go:196] Error preparing data for projected volume kube-api-access-2ddn6 for pod openshift-network-diagnostics/network-check-target-rvdsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:15.788210 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:15.788106    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6 podName:b34e9272-5563-42b1-acce-2e650f738f69 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:16.788087358 +0000 UTC m=+5.046530519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ddn6" (UniqueName: "kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6") pod "network-check-target-rvdsz" (UID: "b34e9272-5563-42b1-acce-2e650f738f69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:16.011619 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.011478    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:55:15 +0000 UTC" deadline="2028-04-20 14:28:45.411763994 +0000 UTC"
Jul 21 18:00:16.011619 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.011546    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15332h28m29.400221429s"
Jul 21 18:00:16.096172 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.096144    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:16.096310 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.096261    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:16.105145 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.105114    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2q8df" event={"ID":"fdfba343-022f-434b-b69a-64677d0384c5","Type":"ContainerStarted","Data":"b1a1b49fc13f07fcf1cd6d42624b32e96195fc174fd130136b5886858f2ff5f6"}
Jul 21 18:00:16.106426 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.106396    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hkl9f" event={"ID":"548f9041-5dfc-4edb-86e3-0e0f276ac154","Type":"ContainerStarted","Data":"5e96d571d5dd63be2a1eee84dacfbfff0216c781bf92defd96472286f10f9ec8"}
Jul 21 18:00:16.107618 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.107582    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xpnv6" event={"ID":"d858ae26-2013-4bb3-be07-621fa975f0f1","Type":"ContainerStarted","Data":"02a245a16840aa1242e2332169cc69c063a01d023a770159e0af92e18ae050a2"}
Jul 21 18:00:16.108766 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.108732    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerStarted","Data":"8797e3029bbe122fe166cc50830e8ff2e59df99216e81b1305f0398c39124291"}
Jul 21 18:00:16.109976 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.109952    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"429a63d9c2626d1901ecb90a55e85b1c1ce96a3a1ddf0ec7f08786ccc96a0340"}
Jul 21 18:00:16.111027 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.111006    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mq2m" event={"ID":"37d433d8-39ce-40a0-a984-13aeea485f20","Type":"ContainerStarted","Data":"69359d8fbe62f9dc8f6c6cbab3d5f45ff259ea9c406ac8cd333d4394eb6fc257"}
Jul 21 18:00:16.112190 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.112140    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vss4j" event={"ID":"e588e98b-3542-4fdd-afa9-fb8b1e3d787e","Type":"ContainerStarted","Data":"bc0156369a9b3482e51a221078f78993348c2e347d254912bff535db60b9fffb"}
Jul 21 18:00:16.113359 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.113335    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cvv4k" event={"ID":"de361996-b492-47df-8f47-24e14804473e","Type":"ContainerStarted","Data":"456c91fe4af92f46e957f4b87dc941cf6b96d02dadfa9abb2eeee36272e34afb"}
Jul 21 18:00:16.114367 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.114343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw" event={"ID":"bb360206-b696-49f2-a38c-34523120782d","Type":"ContainerStarted","Data":"62974b3821fe464975bbbac47292716e66400c1f5d1e7eaf3ca81d1a6f43af72"}
Jul 21 18:00:16.694923 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.694085    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:16.694923 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.694127    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:16.694923 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.694258    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:16.694923 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.694316    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs podName:0ef3f2e2-9940-4e62-a294-673fb6176efc nodeName:}" failed. No retries permitted until 2026-07-21 18:00:18.694297638 +0000 UTC m=+6.952740818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs") pod "network-metrics-daemon-pkx8c" (UID: "0ef3f2e2-9940-4e62-a294-673fb6176efc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:16.694923 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.694746    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:16.694923 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.694803    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret podName:df1e655a-c0cc-4d13-8716-6a0fe968b90d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:18.694787616 +0000 UTC m=+6.953230777 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret") pod "global-pull-secret-syncer-zqmhv" (UID: "df1e655a-c0cc-4d13-8716-6a0fe968b90d") : object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:16.795459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:16.794732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:16.795459 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.794900    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 18:00:16.795459 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.794921    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 18:00:16.795459 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.794935    2578 projected.go:196] Error preparing data for projected volume kube-api-access-2ddn6 for pod openshift-network-diagnostics/network-check-target-rvdsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:16.795459 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:16.795001    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6 podName:b34e9272-5563-42b1-acce-2e650f738f69 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:18.794981913 +0000 UTC m=+7.053425071 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ddn6" (UniqueName: "kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6") pod "network-check-target-rvdsz" (UID: "b34e9272-5563-42b1-acce-2e650f738f69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:17.012167 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:17.011867    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:55:15 +0000 UTC" deadline="2028-01-03 04:29:37.950492645 +0000 UTC"
Jul 21 18:00:17.012167 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:17.012123    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12730h29m20.938377335s"
Jul 21 18:00:17.099663 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:17.098094    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:17.099663 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:17.098248    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:17.099663 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:17.098107    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:17.099663 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:17.098402    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:17.123282 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:17.123239    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" event={"ID":"fd118746fb461e4f1c5de51b2e36feed","Type":"ContainerStarted","Data":"d000f80b73cb77182211134bd1e55ec81b8741bc13a7f4d63e5082d7255a69bd"}
Jul 21 18:00:17.139427 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:17.139362    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-73.ec2.internal" podStartSLOduration=3.139342428 podStartE2EDuration="3.139342428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:17.13920803 +0000 UTC m=+5.397651211" watchObservedRunningTime="2026-07-21 18:00:17.139342428 +0000 UTC m=+5.397785612"
Jul 21 18:00:18.097014 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:18.096756    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:18.097014 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.096929    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:18.131395 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:18.131356    2578 generic.go:356] "Generic (PLEG): container finished" podID="fba785e28d682960c2716b1944cd9f41" containerID="346478a6bf4f95cda0f57e30841c83f725292509302b84d192c48403eb20d580" exitCode=0
Jul 21 18:00:18.132330 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:18.132303    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" event={"ID":"fba785e28d682960c2716b1944cd9f41","Type":"ContainerDied","Data":"346478a6bf4f95cda0f57e30841c83f725292509302b84d192c48403eb20d580"}
Jul 21 18:00:18.714644 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:18.713883    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:18.714644 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:18.713928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:18.714644 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.714058    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:18.714644 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.714116    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs podName:0ef3f2e2-9940-4e62-a294-673fb6176efc nodeName:}" failed. No retries permitted until 2026-07-21 18:00:22.714096911 +0000 UTC m=+10.972540075 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs") pod "network-metrics-daemon-pkx8c" (UID: "0ef3f2e2-9940-4e62-a294-673fb6176efc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:18.714644 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.714504    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:18.714644 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.714554    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret podName:df1e655a-c0cc-4d13-8716-6a0fe968b90d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:22.714539445 +0000 UTC m=+10.972982609 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret") pod "global-pull-secret-syncer-zqmhv" (UID: "df1e655a-c0cc-4d13-8716-6a0fe968b90d") : object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:18.815102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:18.815045    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:18.815297 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.815227    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 18:00:18.815297 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.815248    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 18:00:18.815297 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.815260    2578 projected.go:196] Error preparing data for projected volume kube-api-access-2ddn6 for pod openshift-network-diagnostics/network-check-target-rvdsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:18.815473 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:18.815320    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6 podName:b34e9272-5563-42b1-acce-2e650f738f69 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:22.81530224 +0000 UTC m=+11.073745401 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ddn6" (UniqueName: "kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6") pod "network-check-target-rvdsz" (UID: "b34e9272-5563-42b1-acce-2e650f738f69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:19.097100 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:19.096533    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:19.097100 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:19.096700    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:19.097100 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:19.096531    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:19.097100 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:19.096909    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:20.095981 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:20.095949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:20.096165 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:20.096120    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:21.096266 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:21.096149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:21.096717 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:21.096149    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:21.096717 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:21.096345    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:21.096717 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:21.096401    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:22.096791 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:22.096667    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:22.097307 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.096809    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:22.752016 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:22.751936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:22.752016 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:22.751991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:22.752255 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.752143    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:22.752255 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.752203    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs podName:0ef3f2e2-9940-4e62-a294-673fb6176efc nodeName:}" failed. No retries permitted until 2026-07-21 18:00:30.752185132 +0000 UTC m=+19.010628294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs") pod "network-metrics-daemon-pkx8c" (UID: "0ef3f2e2-9940-4e62-a294-673fb6176efc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:22.752372 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.752267    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:22.752372 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.752302    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret podName:df1e655a-c0cc-4d13-8716-6a0fe968b90d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:30.752291056 +0000 UTC m=+19.010734218 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret") pod "global-pull-secret-syncer-zqmhv" (UID: "df1e655a-c0cc-4d13-8716-6a0fe968b90d") : object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:22.852837 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:22.852786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:22.852989 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.852955    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 18:00:22.852989 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.852976    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 18:00:22.853068 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.852990    2578 projected.go:196] Error preparing data for projected volume kube-api-access-2ddn6 for pod openshift-network-diagnostics/network-check-target-rvdsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:22.853068 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:22.853058    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6 podName:b34e9272-5563-42b1-acce-2e650f738f69 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:30.853039002 +0000 UTC m=+19.111482167 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ddn6" (UniqueName: "kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6") pod "network-check-target-rvdsz" (UID: "b34e9272-5563-42b1-acce-2e650f738f69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:23.097410 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:23.097337    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:23.097851 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:23.097450    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:23.097851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:23.097837    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:23.097975 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:23.097931    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:24.096711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:24.096441    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:24.096711 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:24.096600    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:25.096035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:25.096001    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:25.096467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:25.096111    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:25.096467 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:25.096302    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:25.096467 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:25.096327    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:26.096371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:26.096323    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:26.096789 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:26.096483    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:27.096404 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:27.096370    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:27.096879 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:27.096476    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:27.096879 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:27.096530    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:27.096879 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:27.096625    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:28.096622 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:28.096574    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:28.097058 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:28.096700    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:29.095964 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:29.095930    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:29.096133 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:29.096054    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:29.096133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:29.096109    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:29.096252 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:29.096228    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:30.096162 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:30.096134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:30.096531 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.096239    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:30.810779 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:30.810735    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:30.810958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:30.810790    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:30.810958 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.810913    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:30.811070 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.810976    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs podName:0ef3f2e2-9940-4e62-a294-673fb6176efc nodeName:}" failed. No retries permitted until 2026-07-21 18:00:46.810959521 +0000 UTC m=+35.069402678 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs") pod "network-metrics-daemon-pkx8c" (UID: "0ef3f2e2-9940-4e62-a294-673fb6176efc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:30.811070 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.810912    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:30.811150 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.811078    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret podName:df1e655a-c0cc-4d13-8716-6a0fe968b90d nodeName:}" failed. No retries permitted until 2026-07-21 18:00:46.811057614 +0000 UTC m=+35.069500786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret") pod "global-pull-secret-syncer-zqmhv" (UID: "df1e655a-c0cc-4d13-8716-6a0fe968b90d") : object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:30.912058 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:30.912013    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:30.912220 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.912163    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 18:00:30.912220 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.912188    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 18:00:30.912220 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.912200    2578 projected.go:196] Error preparing data for projected volume kube-api-access-2ddn6 for pod openshift-network-diagnostics/network-check-target-rvdsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:30.912389 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:30.912254    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6 podName:b34e9272-5563-42b1-acce-2e650f738f69 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:46.912240512 +0000 UTC m=+35.170683683 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ddn6" (UniqueName: "kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6") pod "network-check-target-rvdsz" (UID: "b34e9272-5563-42b1-acce-2e650f738f69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:31.096587 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:31.096501    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:31.097005 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:31.096508    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:31.097005 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:31.096653    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:31.097005 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:31.096735    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:32.095759 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:32.095720    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:32.095939 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:32.095841    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:33.097140 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:33.097075    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:33.097611 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:33.097206    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:33.097611 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:33.097274    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:33.097611 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:33.097383    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:34.095895 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:34.095859    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:34.096064 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:34.095995    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:35.096376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.096195    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:35.096878 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.096288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:35.096878 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:35.096480    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:35.096878 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:35.096544    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:35.167868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.167835    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"ec2c17a940ebc1869eb66fb2e6825476bdbcd7a38c9034d4ad85229592e616ed"}
Jul 21 18:00:35.169086 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.169063    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2mq2m" event={"ID":"37d433d8-39ce-40a0-a984-13aeea485f20","Type":"ContainerStarted","Data":"705a82fc86a359baf30e30fe8283bd4b3e64b28c15654d756495681736230070"}
Jul 21 18:00:35.173279 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.173254    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vss4j" event={"ID":"e588e98b-3542-4fdd-afa9-fb8b1e3d787e","Type":"ContainerStarted","Data":"7a9f5464c2dddab04b51b56596cdde084dbdd1a3167351c1b804749ef9a261f1"}
Jul 21 18:00:35.174799 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.174773    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cvv4k" event={"ID":"de361996-b492-47df-8f47-24e14804473e","Type":"ContainerStarted","Data":"66e6b0e6e7ff1f48311856a6f77fc4b5511fa15ddd3f2fe51d4ad87d61aa4fb6"}
Jul 21 18:00:35.176330 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.176286    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw" event={"ID":"bb360206-b696-49f2-a38c-34523120782d","Type":"ContainerStarted","Data":"1c9dc260f3901ce08bd566529c90dc310391358a934c44a12408589ea1c5d040"}
Jul 21 18:00:35.177711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.177664    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2q8df" event={"ID":"fdfba343-022f-434b-b69a-64677d0384c5","Type":"ContainerStarted","Data":"f453fc0808a81e73cc0cffaa67f534e9c8b008dc4e467ca7909505e8f32f596c"}
Jul 21 18:00:35.179532 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.179510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" event={"ID":"fba785e28d682960c2716b1944cd9f41","Type":"ContainerStarted","Data":"3344964b0aaef3948c06908ba865790b843a58c212b3b1d16175f358f4529f12"}
Jul 21 18:00:35.180783 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.180763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xpnv6" event={"ID":"d858ae26-2013-4bb3-be07-621fa975f0f1","Type":"ContainerStarted","Data":"52c6643f7bf81b6092b667b0f3de227dde674bf113278b911c5286dedf0381bc"}
Jul 21 18:00:35.182066 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.182039    2578 generic.go:356] "Generic (PLEG): container finished" podID="0710eca1-fe6f-4b08-a308-1dbe4e70619e" containerID="f6235ecf5da5c029aaf6a86d65a1ab87f291b28dfc3216c2d9eca04d0cbf9f15" exitCode=0
Jul 21 18:00:35.182158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.182076    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerDied","Data":"f6235ecf5da5c029aaf6a86d65a1ab87f291b28dfc3216c2d9eca04d0cbf9f15"}
Jul 21 18:00:35.260144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.260104    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2mq2m" podStartSLOduration=2.83593335 podStartE2EDuration="22.260077063s" totalImagesPullingTime="19.424143713s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:35.252478981 +0000 UTC m=+23.510922160" watchObservedRunningTime="2026-07-21 18:00:35.260077063 +0000 UTC m=+23.518520221"
Jul 21 18:00:35.289365 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.289326    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vss4j" podStartSLOduration=2.854821398 podStartE2EDuration="22.289311635s" totalImagesPullingTime="19.434490237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:35.288895614 +0000 UTC m=+23.547338796" watchObservedRunningTime="2026-07-21 18:00:35.289311635 +0000 UTC m=+23.547754815"
Jul 21 18:00:35.307155 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.307040    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-73.ec2.internal" podStartSLOduration=21.307027171 podStartE2EDuration="21.307027171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:35.306878831 +0000 UTC m=+23.565322010" watchObservedRunningTime="2026-07-21 18:00:35.307027171 +0000 UTC m=+23.565470351"
Jul 21 18:00:35.324884 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.324833    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2q8df" podStartSLOduration=2.866896064 podStartE2EDuration="22.324817254s" totalImagesPullingTime="19.45792119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:35.324414092 +0000 UTC m=+23.582857271" watchObservedRunningTime="2026-07-21 18:00:35.324817254 +0000 UTC m=+23.583260435"
Jul 21 18:00:35.349486 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.349450    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cvv4k" podStartSLOduration=2.8813319870000003 podStartE2EDuration="22.349437854s" totalImagesPullingTime="19.468105867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:35.348907551 +0000 UTC m=+23.607350731" watchObservedRunningTime="2026-07-21 18:00:35.349437854 +0000 UTC m=+23.607881033"
Jul 21 18:00:35.406822 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:35.406781    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xpnv6" podStartSLOduration=7.673046802 podStartE2EDuration="22.406765549s" totalImagesPullingTime="14.733718747s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:35.406364994 +0000 UTC m=+23.664808175" watchObservedRunningTime="2026-07-21 18:00:35.406765549 +0000 UTC m=+23.665208726"
Jul 21 18:00:36.096561 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.096490    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:36.097189 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:36.096625    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:36.188525 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.188503    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:00:36.188886 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.188860    2578 generic.go:356] "Generic (PLEG): container finished" podID="f3be15ef-cbc8-43ee-862b-c8b341d07fb4" containerID="ce33b5dba4959e5a16a666645d084606809394a09ffabc7af51becb5b82257fe" exitCode=1
Jul 21 18:00:36.188983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.188933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"75e57326d19f67b588530ae0046f144cb7c614479c647e333e63c2df5385fcaf"}
Jul 21 18:00:36.189050 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.189036    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"2a0f8161631a31d69c02b0a14f366c2e4ed6f3b0ee56f1c8ec43721ff5a28c13"}
Jul 21 18:00:36.189116 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.189093    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"78dc821cfb63c25e056273c3ab445cae0d7b8f98ebf807525d09a75e10bb8de1"}
Jul 21 18:00:36.189170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.189142    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"05e1326144b9fa95dcf6e4cbd5533e894e2a8c82e5facddc57a6c8f70d3b3513"}
Jul 21 18:00:36.189225 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.189189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerDied","Data":"ce33b5dba4959e5a16a666645d084606809394a09ffabc7af51becb5b82257fe"}
Jul 21 18:00:36.190422 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.190396    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hkl9f" event={"ID":"548f9041-5dfc-4edb-86e3-0e0f276ac154","Type":"ContainerStarted","Data":"3e0f7371ea0dad9068be4a2ab7d564285a62c6cc024bb512830ee7d7c242a65a"}
Jul 21 18:00:36.214688 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.214648    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hkl9f" podStartSLOduration=3.766936732 podStartE2EDuration="23.214636058s" totalImagesPullingTime="19.447699326s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:36.214364352 +0000 UTC m=+24.472807532" watchObservedRunningTime="2026-07-21 18:00:36.214636058 +0000 UTC m=+24.473079238"
Jul 21 18:00:36.276734 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:36.276702    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 18:00:37.035329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:37.035209    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T18:00:36.276719117Z","UUID":"0209a92a-abfd-4787-9f67-64a654f4420f","Handler":null,"Name":"","Endpoint":""}
Jul 21 18:00:37.036853 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:37.036833    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 18:00:37.037024 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:37.036888    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 18:00:37.096861 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:37.096827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:37.096861 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:37.096854    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:37.097382 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:37.096994    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:37.097382 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:37.097117    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:37.195398 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:37.195357    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw" event={"ID":"bb360206-b696-49f2-a38c-34523120782d","Type":"ContainerStarted","Data":"024b90221295456ac95b17e514b4d330dfe94b1dea43cbec6cca4bec0e40119e"}
Jul 21 18:00:38.096735 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:38.096666    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:38.096949 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:38.096788    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:38.201328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:38.201299    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:00:38.201758 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:38.201731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"509cbf62f8bb97717d02cceb0519edc46aeaa9a68fdba75f734573b3a6e26e85"}
Jul 21 18:00:38.203973 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:38.203946    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw" event={"ID":"bb360206-b696-49f2-a38c-34523120782d","Type":"ContainerStarted","Data":"285836909de5a8eeb5a0d07c3608fbed997e13ed3153097240d8306ffc08f4ab"}
Jul 21 18:00:38.223428 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:38.223389    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lckjw" podStartSLOduration=3.059303941 podStartE2EDuration="25.223376392s" totalImagesPullingTime="22.164072451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:38.223271417 +0000 UTC m=+26.481714596" watchObservedRunningTime="2026-07-21 18:00:38.223376392 +0000 UTC m=+26.481819571"
Jul 21 18:00:39.096646 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:39.096440    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:39.096912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:39.096505    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:39.096912 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:39.096741    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:39.096912 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:39.096813    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:39.297725 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:39.297691    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:39.298341 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:39.298322    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:40.096787 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.096769    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:40.096887 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:40.096870    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:40.210797 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.210624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerStarted","Data":"29bc9ec7d17f5b36b1a572955dfc29279d1374062933131daf365984c99a01fd"}
Jul 21 18:00:40.214281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.214258    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:00:40.214650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.214622    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"4d72f3eeaaaccb0fa08e072914cecdddbbbc4b7826b646868b16ff54a4d5b46c"}
Jul 21 18:00:40.214911 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.214893    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:40.215181 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.215149    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:40.215579 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.215300    2578 scope.go:117] "RemoveContainer" containerID="ce33b5dba4959e5a16a666645d084606809394a09ffabc7af51becb5b82257fe"
Jul 21 18:00:40.215579 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.215376    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vss4j"
Jul 21 18:00:40.230833 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:40.230814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:41.096147 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.096114    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:41.096305 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:41.096256    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:41.096356 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.096307    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:41.096404 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:41.096385    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:41.219432 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.219345    2578 generic.go:356] "Generic (PLEG): container finished" podID="0710eca1-fe6f-4b08-a308-1dbe4e70619e" containerID="29bc9ec7d17f5b36b1a572955dfc29279d1374062933131daf365984c99a01fd" exitCode=0
Jul 21 18:00:41.219895 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.219434    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerDied","Data":"29bc9ec7d17f5b36b1a572955dfc29279d1374062933131daf365984c99a01fd"}
Jul 21 18:00:41.223857 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.223830    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:00:41.224308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.224266    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" event={"ID":"f3be15ef-cbc8-43ee-862b-c8b341d07fb4","Type":"ContainerStarted","Data":"9ef174fc6e697ca59d6f9807c3cdf88eed5dd41f9d6f4cc261b51714ab97644c"}
Jul 21 18:00:41.224403 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.224341    2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 18:00:41.224583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.224551    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:41.252370 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.252339    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:41.296228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:41.296181    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2" podStartSLOduration=8.695358193 podStartE2EDuration="28.296166037s" totalImagesPullingTime="19.600807844s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:41.294096975 +0000 UTC m=+29.552540156" watchObservedRunningTime="2026-07-21 18:00:41.296166037 +0000 UTC m=+29.554609232"
Jul 21 18:00:42.095816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.095786    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:42.096016 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:42.095896    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:42.165501 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.165465    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkx8c"]
Jul 21 18:00:42.166109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.166087    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zqmhv"]
Jul 21 18:00:42.166193 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.166182    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:42.166283 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:42.166267    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:42.167865 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.167843    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rvdsz"]
Jul 21 18:00:42.167992 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.167925    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:42.168051 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:42.168025    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:42.226389 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.226319    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:42.226810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:42.226403    2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 18:00:42.226850 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:42.226807    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:43.184545 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:43.184511    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:00:43.229951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:43.229917    2578 generic.go:356] "Generic (PLEG): container finished" podID="0710eca1-fe6f-4b08-a308-1dbe4e70619e" containerID="11fd5901c8f0cecc77fab3e35da076077926bc3db2baa1195eb9c62340f1a570" exitCode=0
Jul 21 18:00:43.230305 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:43.230001    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerDied","Data":"11fd5901c8f0cecc77fab3e35da076077926bc3db2baa1195eb9c62340f1a570"}
Jul 21 18:00:44.096950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:44.096766    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:44.097102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:44.096770    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:44.097102 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:44.097043    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:44.097102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:44.096816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:44.097262 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:44.097101    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:44.097262 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:44.097173    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:46.096574 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.096484    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:46.096574 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.096532    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:46.096574 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.096532    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:46.097091 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.096608    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rvdsz" podUID="b34e9272-5563-42b1-acce-2e650f738f69"
Jul 21 18:00:46.097091 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.096686    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-pkx8c" podUID="0ef3f2e2-9940-4e62-a294-673fb6176efc"
Jul 21 18:00:46.097091 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.096748    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zqmhv" podUID="df1e655a-c0cc-4d13-8716-6a0fe968b90d"
Jul 21 18:00:46.238573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.238540    2578 generic.go:356] "Generic (PLEG): container finished" podID="0710eca1-fe6f-4b08-a308-1dbe4e70619e" containerID="b6ae5f1a623ccf5183d5b2c9a30b17b1c86eb9510e5cd76d9e92be1b0bd2e3fa" exitCode=0
Jul 21 18:00:46.238750 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.238634    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerDied","Data":"b6ae5f1a623ccf5183d5b2c9a30b17b1c86eb9510e5cd76d9e92be1b0bd2e3fa"}
Jul 21 18:00:46.822950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.822910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:46.822950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.822947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:46.823176 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.823063    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:46.823176 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.823067    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:46.823176 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.823114    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs podName:0ef3f2e2-9940-4e62-a294-673fb6176efc nodeName:}" failed. No retries permitted until 2026-07-21 18:01:18.823101048 +0000 UTC m=+67.081544206 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs") pod "network-metrics-daemon-pkx8c" (UID: "0ef3f2e2-9940-4e62-a294-673fb6176efc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 18:00:46.823176 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.823132    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret podName:df1e655a-c0cc-4d13-8716-6a0fe968b90d nodeName:}" failed. No retries permitted until 2026-07-21 18:01:18.823122872 +0000 UTC m=+67.081566030 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret") pod "global-pull-secret-syncer-zqmhv" (UID: "df1e655a-c0cc-4d13-8716-6a0fe968b90d") : object "kube-system"/"original-pull-secret" not registered
Jul 21 18:00:46.924144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:46.924115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:46.924329 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.924278    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 18:00:46.924329 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.924302    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 18:00:46.924329 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.924314    2578 projected.go:196] Error preparing data for projected volume kube-api-access-2ddn6 for pod openshift-network-diagnostics/network-check-target-rvdsz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:46.924494 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:46.924376    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6 podName:b34e9272-5563-42b1-acce-2e650f738f69 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:18.924357966 +0000 UTC m=+67.182801138 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2ddn6" (UniqueName: "kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6") pod "network-check-target-rvdsz" (UID: "b34e9272-5563-42b1-acce-2e650f738f69") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 18:00:47.475657 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.475624    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-73.ec2.internal" event="NodeReady"
Jul 21 18:00:47.476218 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.475791    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 18:00:47.528007 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.527920    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b5f4f5784-9vhpv"]
Jul 21 18:00:47.530024 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.529996    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.533802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.533776    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 21 18:00:47.533915 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.533782    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 21 18:00:47.534118 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.534094    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 21 18:00:47.534203 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.534151    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7t5gz\""
Jul 21 18:00:47.537954 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.537931    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cbplt"]
Jul 21 18:00:47.539903 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.539884    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.544851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.543616    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 18:00:47.544851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.544117    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 18:00:47.544851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.544757    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pqxk6\""
Jul 21 18:00:47.545832 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.545807    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 21 18:00:47.546694 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.546675    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sltvh"]
Jul 21 18:00:47.548472 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.548451    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.551364 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.551347    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5f4f5784-9vhpv"]
Jul 21 18:00:47.553697 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.553671    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wbjcp\""
Jul 21 18:00:47.555371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.554924    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 18:00:47.555371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.554972    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cbplt"]
Jul 21 18:00:47.555371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.555168    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 18:00:47.555371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.555217    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 18:00:47.561394 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.561291    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sltvh"]
Jul 21 18:00:47.630449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-bound-sa-token\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.630449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.630684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg9f4\" (UniqueName: \"kubernetes.io/projected/96003adc-b20f-487f-b330-358bdf2473d0-kube-api-access-mg9f4\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.630684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630604    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-registry-certificates\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.630684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630645    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d508cf76-c00c-42e6-bd27-88879c52c149-cert\") pod \"ingress-canary-sltvh\" (UID: \"d508cf76-c00c-42e6-bd27-88879c52c149\") " pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.630684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btkpr\" (UniqueName: \"kubernetes.io/projected/d508cf76-c00c-42e6-bd27-88879c52c149-kube-api-access-btkpr\") pod \"ingress-canary-sltvh\" (UID: \"d508cf76-c00c-42e6-bd27-88879c52c149\") " pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.630859 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4wvk\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-kube-api-access-t4wvk\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.630859 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630750    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8407270-3386-433d-920b-9266dea58788-ca-trust-extracted\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.630859 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96003adc-b20f-487f-b330-358bdf2473d0-tmp-dir\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.630859 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.631046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-image-registry-private-configuration\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.631046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-trusted-ca\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.631046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96003adc-b20f-487f-b330-358bdf2473d0-config-volume\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.631046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.630963    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-installation-pull-secrets\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.731854 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.731818    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4wvk\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-kube-api-access-t4wvk\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.731854 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.731871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8407270-3386-433d-920b-9266dea58788-ca-trust-extracted\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.731898    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96003adc-b20f-487f-b330-358bdf2473d0-tmp-dir\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.731930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.731963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-image-registry-private-configuration\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.731990    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-trusted-ca\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732016    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96003adc-b20f-487f-b330-358bdf2473d0-config-volume\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732049    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-installation-pull-secrets\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-bound-sa-token\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:47.732097    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 18:00:47.732109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732112    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.732486 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:47.732120    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5f4f5784-9vhpv: secret "image-registry-tls" not found
Jul 21 18:00:47.732486 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg9f4\" (UniqueName: \"kubernetes.io/projected/96003adc-b20f-487f-b330-358bdf2473d0-kube-api-access-mg9f4\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.732486 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:47.732180    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls podName:a8407270-3386-433d-920b-9266dea58788 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:48.232159711 +0000 UTC m=+36.490602874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls") pod "image-registry-5b5f4f5784-9vhpv" (UID: "a8407270-3386-433d-920b-9266dea58788") : secret "image-registry-tls" not found
Jul 21 18:00:47.732486 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-registry-certificates\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732486 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d508cf76-c00c-42e6-bd27-88879c52c149-cert\") pod \"ingress-canary-sltvh\" (UID: \"d508cf76-c00c-42e6-bd27-88879c52c149\") " pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.732486 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btkpr\" (UniqueName: \"kubernetes.io/projected/d508cf76-c00c-42e6-bd27-88879c52c149-kube-api-access-btkpr\") pod \"ingress-canary-sltvh\" (UID: \"d508cf76-c00c-42e6-bd27-88879c52c149\") " pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.732486 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732317    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/96003adc-b20f-487f-b330-358bdf2473d0-tmp-dir\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.732838 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.732527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8407270-3386-433d-920b-9266dea58788-ca-trust-extracted\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.732838 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:47.732580    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 18:00:47.732838 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:47.732662    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls podName:96003adc-b20f-487f-b330-358bdf2473d0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:48.23264667 +0000 UTC m=+36.491089849 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls") pod "dns-default-cbplt" (UID: "96003adc-b20f-487f-b330-358bdf2473d0") : secret "dns-default-metrics-tls" not found
Jul 21 18:00:47.733438 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.733381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/96003adc-b20f-487f-b330-358bdf2473d0-config-volume\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.733684 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.733662    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-trusted-ca\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.733940 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.733921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-registry-certificates\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.737286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.737267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-image-registry-private-configuration\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.737286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.737274    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d508cf76-c00c-42e6-bd27-88879c52c149-cert\") pod \"ingress-canary-sltvh\" (UID: \"d508cf76-c00c-42e6-bd27-88879c52c149\") " pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.737423 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.737294    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-installation-pull-secrets\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.755251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.755219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg9f4\" (UniqueName: \"kubernetes.io/projected/96003adc-b20f-487f-b330-358bdf2473d0-kube-api-access-mg9f4\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:47.755364 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.755298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-bound-sa-token\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.757007 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.756989    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btkpr\" (UniqueName: \"kubernetes.io/projected/d508cf76-c00c-42e6-bd27-88879c52c149-kube-api-access-btkpr\") pod \"ingress-canary-sltvh\" (UID: \"d508cf76-c00c-42e6-bd27-88879c52c149\") " pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.757570 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.757550    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4wvk\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-kube-api-access-t4wvk\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:47.759645 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.759627    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hnv29"]
Jul 21 18:00:47.763109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.763089    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:47.766799 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.766701    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 18:00:47.766799 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.766726    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 18:00:47.768084 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.768060    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-gzbvh\""
Jul 21 18:00:47.768195 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.768120    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 18:00:47.768271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.768194    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 18:00:47.778229 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.778170    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnv29"]
Jul 21 18:00:47.862738 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.862705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sltvh"
Jul 21 18:00:47.934786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.934572    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-crio-socket\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:47.934786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.934618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-data-volume\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:47.934786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.934653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:47.934786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.934669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:47.934786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:47.934721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7ldb\" (UniqueName: \"kubernetes.io/projected/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-api-access-p7ldb\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.023116 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.022860    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sltvh"]
Jul 21 18:00:48.028116 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:48.028091    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd508cf76_c00c_42e6_bd27_88879c52c149.slice/crio-0624ae4b363d9d4f137cb7676aa5294161067d14202378f34543487dd86fdac7 WatchSource:0}: Error finding container 0624ae4b363d9d4f137cb7676aa5294161067d14202378f34543487dd86fdac7: Status 404 returned error can't find the container with id 0624ae4b363d9d4f137cb7676aa5294161067d14202378f34543487dd86fdac7
Jul 21 18:00:48.036077 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.036050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-crio-socket\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.036157 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.036091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-data-volume\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.036157 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.036137    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.036258 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.036165    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.036310 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.036258    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 18:00:48.036310 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.036278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-crio-socket\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.036416 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.036312    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7ldb\" (UniqueName: \"kubernetes.io/projected/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-api-access-p7ldb\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.036416 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.036319    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:48.536300116 +0000 UTC m=+36.794743292 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : configmap "kube-rbac-proxy" not found
Jul 21 18:00:48.036416 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.036375    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:48.036563 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.036416    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:48.536403741 +0000 UTC m=+36.794846902 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:48.036563 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.036490    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-data-volume\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.055751 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.055727    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7ldb\" (UniqueName: \"kubernetes.io/projected/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-api-access-p7ldb\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.095938 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.095902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:00:48.096092 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.095949    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:00:48.096092 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.096012    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:00:48.099246 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.099226    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 18:00:48.099382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.099358    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 18:00:48.099444 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.099404    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jg5zx\""
Jul 21 18:00:48.100433 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.100412    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 18:00:48.100527 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.100415    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vznt9\""
Jul 21 18:00:48.100577 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.100425    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 18:00:48.238030 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.237988    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:48.238217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.238065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:48.238217 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.238164    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 18:00:48.238217 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.238192    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 18:00:48.238217 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.238215    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5f4f5784-9vhpv: secret "image-registry-tls" not found
Jul 21 18:00:48.238380 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.238266    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls podName:96003adc-b20f-487f-b330-358bdf2473d0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:49.238235811 +0000 UTC m=+37.496678982 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls") pod "dns-default-cbplt" (UID: "96003adc-b20f-487f-b330-358bdf2473d0") : secret "dns-default-metrics-tls" not found
Jul 21 18:00:48.238380 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.238286    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls podName:a8407270-3386-433d-920b-9266dea58788 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:49.238276976 +0000 UTC m=+37.496720136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls") pod "image-registry-5b5f4f5784-9vhpv" (UID: "a8407270-3386-433d-920b-9266dea58788") : secret "image-registry-tls" not found
Jul 21 18:00:48.244302 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.244272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sltvh" event={"ID":"d508cf76-c00c-42e6-bd27-88879c52c149","Type":"ContainerStarted","Data":"0624ae4b363d9d4f137cb7676aa5294161067d14202378f34543487dd86fdac7"}
Jul 21 18:00:48.541838 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.541788    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.542525 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:48.541856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:48.542525 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.541943    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 18:00:48.542525 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.542030    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:49.542010009 +0000 UTC m=+37.800453187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : configmap "kube-rbac-proxy" not found
Jul 21 18:00:48.542525 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.542066    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:48.542525 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:48.542145    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:49.542113791 +0000 UTC m=+37.800556953 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:49.246718 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.246670    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:49.246903 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.246816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:49.246978 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.246936    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 18:00:49.246978 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.246951    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5f4f5784-9vhpv: secret "image-registry-tls" not found
Jul 21 18:00:49.247084 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.246992    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 18:00:49.247084 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.247016    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls podName:a8407270-3386-433d-920b-9266dea58788 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.246997682 +0000 UTC m=+39.505440843 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls") pod "image-registry-5b5f4f5784-9vhpv" (UID: "a8407270-3386-433d-920b-9266dea58788") : secret "image-registry-tls" not found
Jul 21 18:00:49.247084 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.247056    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls podName:96003adc-b20f-487f-b330-358bdf2473d0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.247035501 +0000 UTC m=+39.505478673 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls") pod "dns-default-cbplt" (UID: "96003adc-b20f-487f-b330-358bdf2473d0") : secret "dns-default-metrics-tls" not found
Jul 21 18:00:49.252946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.252724    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-l2fs9"]
Jul 21 18:00:49.255177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.255153    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.258729 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.258704    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 21 18:00:49.258896 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.258704    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 21 18:00:49.259583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.259558    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 21 18:00:49.259683 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.259638    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 21 18:00:49.260277 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.260258    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-85jgl\""
Jul 21 18:00:49.269011 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.268989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-l2fs9"]
Jul 21 18:00:49.347510 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.347471    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7393cd1a-3522-4927-b2ca-9624140dae6f-signing-cabundle\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.347697 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.347612    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7393cd1a-3522-4927-b2ca-9624140dae6f-signing-key\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.347697 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.347653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4qw9\" (UniqueName: \"kubernetes.io/projected/7393cd1a-3522-4927-b2ca-9624140dae6f-kube-api-access-c4qw9\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.448495 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.448449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7393cd1a-3522-4927-b2ca-9624140dae6f-signing-cabundle\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.448701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.448564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7393cd1a-3522-4927-b2ca-9624140dae6f-signing-key\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.448701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.448625    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4qw9\" (UniqueName: \"kubernetes.io/projected/7393cd1a-3522-4927-b2ca-9624140dae6f-kube-api-access-c4qw9\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.449246 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.449224    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/7393cd1a-3522-4927-b2ca-9624140dae6f-signing-cabundle\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.451908 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.451877    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/7393cd1a-3522-4927-b2ca-9624140dae6f-signing-key\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.467968 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.467942    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4qw9\" (UniqueName: \"kubernetes.io/projected/7393cd1a-3522-4927-b2ca-9624140dae6f-kube-api-access-c4qw9\") pod \"service-ca-58598695f7-l2fs9\" (UID: \"7393cd1a-3522-4927-b2ca-9624140dae6f\") " pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.550227 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.549999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:49.550227 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.550048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:49.550227 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.550124    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 18:00:49.550227 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.550196    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:49.550227 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.550213    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.550192587 +0000 UTC m=+39.808635751 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : configmap "kube-rbac-proxy" not found
Jul 21 18:00:49.550887 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:49.550256    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:51.550240195 +0000 UTC m=+39.808683398 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:49.568940 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.568893    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-l2fs9"
Jul 21 18:00:49.909428 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:49.909397    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2q8df_fdfba343-022f-434b-b69a-64677d0384c5/dns-node-resolver/0.log"
Jul 21 18:00:50.496969 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:50.496897    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-l2fs9"]
Jul 21 18:00:51.135630 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:00:51.135584    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7393cd1a_3522_4927_b2ca_9624140dae6f.slice/crio-0847c9fee5f8b084638140a3638bcae55d5f89843fa22e84b40a69fb760f491d WatchSource:0}: Error finding container 0847c9fee5f8b084638140a3638bcae55d5f89843fa22e84b40a69fb760f491d: Status 404 returned error can't find the container with id 0847c9fee5f8b084638140a3638bcae55d5f89843fa22e84b40a69fb760f491d
Jul 21 18:00:51.257358 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:51.257163    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerStarted","Data":"d2665cc1f46b2d8ee0fb7a7ec316e964cc988e502255fa998ee1261a0ed3590a"}
Jul 21 18:00:51.258433 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:51.258407    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-l2fs9" event={"ID":"7393cd1a-3522-4927-b2ca-9624140dae6f","Type":"ContainerStarted","Data":"0847c9fee5f8b084638140a3638bcae55d5f89843fa22e84b40a69fb760f491d"}
Jul 21 18:00:51.268507 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:51.268485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:51.268630 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:51.268578    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:51.268689 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.268665    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 18:00:51.268745 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.268709    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 18:00:51.268745 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.268730    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5f4f5784-9vhpv: secret "image-registry-tls" not found
Jul 21 18:00:51.268745 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.268741    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls podName:96003adc-b20f-487f-b330-358bdf2473d0 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:55.268721251 +0000 UTC m=+43.527164412 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls") pod "dns-default-cbplt" (UID: "96003adc-b20f-487f-b330-358bdf2473d0") : secret "dns-default-metrics-tls" not found
Jul 21 18:00:51.268896 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.268787    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls podName:a8407270-3386-433d-920b-9266dea58788 nodeName:}" failed. No retries permitted until 2026-07-21 18:00:55.268768451 +0000 UTC m=+43.527211644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls") pod "image-registry-5b5f4f5784-9vhpv" (UID: "a8407270-3386-433d-920b-9266dea58788") : secret "image-registry-tls" not found
Jul 21 18:00:51.306975 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:51.306944    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xpnv6_d858ae26-2013-4bb3-be07-621fa975f0f1/node-ca/0.log"
Jul 21 18:00:51.573543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:51.573445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:51.573543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:51.573491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:51.573785 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.573602    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 18:00:51.573785 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.573659    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:51.573785 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.573695    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:55.573679004 +0000 UTC m=+43.832122162 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : configmap "kube-rbac-proxy" not found
Jul 21 18:00:51.573785 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:51.573724    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:00:55.57370836 +0000 UTC m=+43.832151537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:52.264654 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:52.264618    2578 generic.go:356] "Generic (PLEG): container finished" podID="0710eca1-fe6f-4b08-a308-1dbe4e70619e" containerID="d2665cc1f46b2d8ee0fb7a7ec316e964cc988e502255fa998ee1261a0ed3590a" exitCode=0
Jul 21 18:00:52.265307 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:52.264708    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerDied","Data":"d2665cc1f46b2d8ee0fb7a7ec316e964cc988e502255fa998ee1261a0ed3590a"}
Jul 21 18:00:52.266286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:52.266255    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sltvh" event={"ID":"d508cf76-c00c-42e6-bd27-88879c52c149","Type":"ContainerStarted","Data":"b4dc1e3c452f8ef1588a4daf71e6de3b14463efc2b7f7bb218c933970c621453"}
Jul 21 18:00:52.312419 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:52.312368    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sltvh" podStartSLOduration=2.11295983 podStartE2EDuration="5.312347989s" totalImagesPullingTime="3.199388159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:52.311929507 +0000 UTC m=+40.570372710" watchObservedRunningTime="2026-07-21 18:00:52.312347989 +0000 UTC m=+40.570791170"
Jul 21 18:00:53.271553 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:53.271520    2578 generic.go:356] "Generic (PLEG): container finished" podID="0710eca1-fe6f-4b08-a308-1dbe4e70619e" containerID="34a2bbaf5ee57e31ac8cf977f3f0b095205b3b78b26010401e913d1e931e2efa" exitCode=0
Jul 21 18:00:53.271944 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:53.271582    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerDied","Data":"34a2bbaf5ee57e31ac8cf977f3f0b095205b3b78b26010401e913d1e931e2efa"}
Jul 21 18:00:54.277326 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:54.277290    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-p9m59" event={"ID":"0710eca1-fe6f-4b08-a308-1dbe4e70619e","Type":"ContainerStarted","Data":"77372b793dfbc3c2875cad7364093411b2d726e6f4aaa00d62199929592b9e7c"}
Jul 21 18:00:54.278868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:54.278846    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-l2fs9" event={"ID":"7393cd1a-3522-4927-b2ca-9624140dae6f","Type":"ContainerStarted","Data":"3be27c416e1bd5f31e8249ec8440f0b023b9149bfc7ef0ddcb3ee68d9f804640"}
Jul 21 18:00:54.309157 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:54.309105    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-p9m59" podStartSLOduration=9.148172957 podStartE2EDuration="41.309090272s" totalImagesPullingTime="32.160917315s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:54.306748281 +0000 UTC m=+42.565191460" watchObservedRunningTime="2026-07-21 18:00:54.309090272 +0000 UTC m=+42.567533452"
Jul 21 18:00:54.325168 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:54.325128    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-l2fs9" podStartSLOduration=3.058648594 podStartE2EDuration="5.325115478s" totalImagesPullingTime="2.266466884s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:00:54.324780327 +0000 UTC m=+42.583223519" watchObservedRunningTime="2026-07-21 18:00:54.325115478 +0000 UTC m=+42.583558659"
Jul 21 18:00:55.305718 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:55.305682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:00:55.306155 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:55.306002    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 18:00:55.306155 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:55.306061    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls podName:96003adc-b20f-487f-b330-358bdf2473d0 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:03.306043646 +0000 UTC m=+51.564486808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls") pod "dns-default-cbplt" (UID: "96003adc-b20f-487f-b330-358bdf2473d0") : secret "dns-default-metrics-tls" not found
Jul 21 18:00:55.306526 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:55.306495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:00:55.306665 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:55.306647    2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 21 18:00:55.306725 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:55.306669    2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b5f4f5784-9vhpv: secret "image-registry-tls" not found
Jul 21 18:00:55.306725 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:55.306717    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls podName:a8407270-3386-433d-920b-9266dea58788 nodeName:}" failed. No retries permitted until 2026-07-21 18:01:03.306701228 +0000 UTC m=+51.565144402 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls") pod "image-registry-5b5f4f5784-9vhpv" (UID: "a8407270-3386-433d-920b-9266dea58788") : secret "image-registry-tls" not found
Jul 21 18:00:55.608763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:55.608675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:55.608763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:55.608710    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:00:55.608994 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:55.608914    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:55.608994 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:00:55.608986    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls podName:f87c1304-0a50-4bf2-8b8f-c3e8cea362ae nodeName:}" failed. No retries permitted until 2026-07-21 18:01:03.608968061 +0000 UTC m=+51.867411231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hnv29" (UID: "f87c1304-0a50-4bf2-8b8f-c3e8cea362ae") : secret "insights-runtime-extractor-tls" not found
Jul 21 18:00:55.609408 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:00:55.609388    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:01:03.373958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.373917    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:01:03.374407 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.374046    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:01:03.377493 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.377459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"image-registry-5b5f4f5784-9vhpv\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") " pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:01:03.377622 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.377464    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/96003adc-b20f-487f-b330-358bdf2473d0-metrics-tls\") pod \"dns-default-cbplt\" (UID: \"96003adc-b20f-487f-b330-358bdf2473d0\") " pod="openshift-dns/dns-default-cbplt"
Jul 21 18:01:03.441475 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.441443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:01:03.451722 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.451692    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cbplt"
Jul 21 18:01:03.604326 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.604287    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b5f4f5784-9vhpv"]
Jul 21 18:01:03.607410 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:03.607383    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8407270_3386_433d_920b_9266dea58788.slice/crio-147c40256765cde2bc007fc0452b9ceff94f7fc7114e12b3c0621088271cc01b WatchSource:0}: Error finding container 147c40256765cde2bc007fc0452b9ceff94f7fc7114e12b3c0621088271cc01b: Status 404 returned error can't find the container with id 147c40256765cde2bc007fc0452b9ceff94f7fc7114e12b3c0621088271cc01b
Jul 21 18:01:03.626059 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.625995    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cbplt"]
Jul 21 18:01:03.631712 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:03.631689    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96003adc_b20f_487f_b330_358bdf2473d0.slice/crio-e7a0df58a77998ba81b5daaa89f1059ec27df2d227312b465575b0b313e59c33 WatchSource:0}: Error finding container e7a0df58a77998ba81b5daaa89f1059ec27df2d227312b465575b0b313e59c33: Status 404 returned error can't find the container with id e7a0df58a77998ba81b5daaa89f1059ec27df2d227312b465575b0b313e59c33
Jul 21 18:01:03.677051 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.677019    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:01:03.679120 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.679102    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f87c1304-0a50-4bf2-8b8f-c3e8cea362ae-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hnv29\" (UID: \"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae\") " pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:01:03.976613 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:03.976561    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hnv29"
Jul 21 18:01:04.124857 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:04.124825    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hnv29"]
Jul 21 18:01:04.128031 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:04.128007    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf87c1304_0a50_4bf2_8b8f_c3e8cea362ae.slice/crio-f9966425741fbc1c0a5514aacb04127f1be33c83a0da4041b0747334379614d8 WatchSource:0}: Error finding container f9966425741fbc1c0a5514aacb04127f1be33c83a0da4041b0747334379614d8: Status 404 returned error can't find the container with id f9966425741fbc1c0a5514aacb04127f1be33c83a0da4041b0747334379614d8
Jul 21 18:01:04.304656 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:04.304570    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv" event={"ID":"a8407270-3386-433d-920b-9266dea58788","Type":"ContainerStarted","Data":"17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73"}
Jul 21 18:01:04.304656 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:04.304632    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv" event={"ID":"a8407270-3386-433d-920b-9266dea58788","Type":"ContainerStarted","Data":"147c40256765cde2bc007fc0452b9ceff94f7fc7114e12b3c0621088271cc01b"}
Jul 21 18:01:04.304841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:04.304678    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:01:04.305872 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:04.305844    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnv29" event={"ID":"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae","Type":"ContainerStarted","Data":"f9966425741fbc1c0a5514aacb04127f1be33c83a0da4041b0747334379614d8"}
Jul 21 18:01:04.306918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:04.306898    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cbplt" event={"ID":"96003adc-b20f-487f-b330-358bdf2473d0","Type":"ContainerStarted","Data":"e7a0df58a77998ba81b5daaa89f1059ec27df2d227312b465575b0b313e59c33"}
Jul 21 18:01:04.329494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:04.329450    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv" podStartSLOduration=35.329438023 podStartE2EDuration="35.329438023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:04.328981511 +0000 UTC m=+52.587424705" watchObservedRunningTime="2026-07-21 18:01:04.329438023 +0000 UTC m=+52.587881203"
Jul 21 18:01:10.330494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:10.330365    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnv29" event={"ID":"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae","Type":"ContainerStarted","Data":"7462e19446e126e4d19ab51e54d4b553b15bffb2b42ec27cbd5c16005d486bba"}
Jul 21 18:01:10.332585 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:10.332471    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cbplt" event={"ID":"96003adc-b20f-487f-b330-358bdf2473d0","Type":"ContainerStarted","Data":"b4578362c5232468768607cefa00ccea178bfe808792fe1b779fbc81677a9efe"}
Jul 21 18:01:10.332585 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:10.332522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cbplt" event={"ID":"96003adc-b20f-487f-b330-358bdf2473d0","Type":"ContainerStarted","Data":"c7d04ff15e4f644ff1e742e61418658147b05dafbdf67b0984bd6ffa35b3a20f"}
Jul 21 18:01:10.332797 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:10.332763    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cbplt"
Jul 21 18:01:10.375648 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:10.375138    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cbplt" podStartSLOduration=17.553206371 podStartE2EDuration="23.375118161s" totalImagesPullingTime="5.82191179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:10.374044806 +0000 UTC m=+58.632487986" watchObservedRunningTime="2026-07-21 18:01:10.375118161 +0000 UTC m=+58.633561343"
Jul 21 18:01:12.347517 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.347476    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"]
Jul 21 18:01:12.349564 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.349541    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"]
Jul 21 18:01:12.349701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.349682    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.351408 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.351389    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"]
Jul 21 18:01:12.351604 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.351556    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.353045 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.353005    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 18:01:12.353176 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.353010    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 18:01:12.353236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.353193    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 18:01:12.353291 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.353234    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 21 18:01:12.353554 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.353529    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.356836 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.356820    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-shrch\""
Jul 21 18:01:12.357298 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.357265    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 21 18:01:12.357402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.357304    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 21 18:01:12.359788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.359771    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 21 18:01:12.360057 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.360037    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 21 18:01:12.360057 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.360037    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 21 18:01:12.369282 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.369259    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"]
Jul 21 18:01:12.370707 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.370686    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"]
Jul 21 18:01:12.374731 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.374667    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"]
Jul 21 18:01:12.417680 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.417640    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5f4f5784-9vhpv"]
Jul 21 18:01:12.453063 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453035    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.453212 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/52da86bd-4022-4f70-b9d9-994fa9aabdf6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.453212 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453184    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26bca50-ef32-4ecb-8800-f2274a4a484b-tmp\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.453306 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c26bca50-ef32-4ecb-8800-f2274a4a484b-klusterlet-config\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.453306 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453261    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-ca\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.453306 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453287    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjcr5\" (UniqueName: \"kubernetes.io/projected/b76a73fc-f769-45ce-af14-3bd2a0ceea87-kube-api-access-kjcr5\") pod \"managed-serviceaccount-addon-agent-7f88476b9b-kw99t\" (UID: \"b76a73fc-f769-45ce-af14-3bd2a0ceea87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.453410 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-hub\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.453449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453412    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-krfzh\" (UniqueName: \"kubernetes.io/projected/52da86bd-4022-4f70-b9d9-994fa9aabdf6-kube-api-access-krfzh\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.453494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453445    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.453494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453473    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhwkj\" (UniqueName: \"kubernetes.io/projected/c26bca50-ef32-4ecb-8800-f2274a4a484b-kube-api-access-nhwkj\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.453586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.453508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b76a73fc-f769-45ce-af14-3bd2a0ceea87-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f88476b9b-kw99t\" (UID: \"b76a73fc-f769-45ce-af14-3bd2a0ceea87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.547694 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.547657    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-58c7b989d5-9k7cr"]
Jul 21 18:01:12.553324 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.553304    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.553841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.553813    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.553950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.553849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhwkj\" (UniqueName: \"kubernetes.io/projected/c26bca50-ef32-4ecb-8800-f2274a4a484b-kube-api-access-nhwkj\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.553950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.553884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b76a73fc-f769-45ce-af14-3bd2a0ceea87-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f88476b9b-kw99t\" (UID: \"b76a73fc-f769-45ce-af14-3bd2a0ceea87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554158    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/52da86bd-4022-4f70-b9d9-994fa9aabdf6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554210    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26bca50-ef32-4ecb-8800-f2274a4a484b-tmp\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c26bca50-ef32-4ecb-8800-f2274a4a484b-klusterlet-config\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554271    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-ca\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjcr5\" (UniqueName: \"kubernetes.io/projected/b76a73fc-f769-45ce-af14-3bd2a0ceea87-kube-api-access-kjcr5\") pod \"managed-serviceaccount-addon-agent-7f88476b9b-kw99t\" (UID: \"b76a73fc-f769-45ce-af14-3bd2a0ceea87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554338    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-hub\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.554810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.554372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-krfzh\" (UniqueName: \"kubernetes.io/projected/52da86bd-4022-4f70-b9d9-994fa9aabdf6-kube-api-access-krfzh\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.555718 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.555692    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c26bca50-ef32-4ecb-8800-f2274a4a484b-tmp\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.556044 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.555993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/52da86bd-4022-4f70-b9d9-994fa9aabdf6-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.557058 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.557000    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/b76a73fc-f769-45ce-af14-3bd2a0ceea87-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7f88476b9b-kw99t\" (UID: \"b76a73fc-f769-45ce-af14-3bd2a0ceea87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.557323 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.557275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.557516 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.557488    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.557901 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.557876    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-ca\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.558657 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.558632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/52da86bd-4022-4f70-b9d9-994fa9aabdf6-hub\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.558764 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.558637    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/c26bca50-ef32-4ecb-8800-f2274a4a484b-klusterlet-config\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.565130 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.565068    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c7b989d5-9k7cr"]
Jul 21 18:01:12.571578 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.571520    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhwkj\" (UniqueName: \"kubernetes.io/projected/c26bca50-ef32-4ecb-8800-f2274a4a484b-kube-api-access-nhwkj\") pod \"klusterlet-addon-workmgr-99d6b5579-2h5bf\" (UID: \"c26bca50-ef32-4ecb-8800-f2274a4a484b\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.571706 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.571572    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjcr5\" (UniqueName: \"kubernetes.io/projected/b76a73fc-f769-45ce-af14-3bd2a0ceea87-kube-api-access-kjcr5\") pod \"managed-serviceaccount-addon-agent-7f88476b9b-kw99t\" (UID: \"b76a73fc-f769-45ce-af14-3bd2a0ceea87\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.572207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.572180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-krfzh\" (UniqueName: \"kubernetes.io/projected/52da86bd-4022-4f70-b9d9-994fa9aabdf6-kube-api-access-krfzh\") pod \"cluster-proxy-proxy-agent-664b78d767-4kjzl\" (UID: \"52da86bd-4022-4f70-b9d9-994fa9aabdf6\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.655112 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655077    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-installation-pull-secrets\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.655275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-trusted-ca\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.655275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655172    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-registry-tls\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.655275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-registry-certificates\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.655275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655234    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhbmd\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-kube-api-access-xhbmd\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.655465 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-bound-sa-token\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.655465 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655337    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-image-registry-private-configuration\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.655465 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.655367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-ca-trust-extracted\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.670071 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.670037    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"
Jul 21 18:01:12.678829 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.678806    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"
Jul 21 18:01:12.692074 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.692049    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:12.755829 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.755794    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-installation-pull-secrets\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.755971 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.755848    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-trusted-ca\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.755971 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.755888    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-registry-tls\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.755971 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.755935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-registry-certificates\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.755971 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.755963    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhbmd\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-kube-api-access-xhbmd\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.756158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.756012    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-bound-sa-token\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.756158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.756039    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-image-registry-private-configuration\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.756158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.756083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-ca-trust-extracted\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.756834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.756785    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-ca-trust-extracted\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.756990 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.756965    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-trusted-ca\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.757607 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.757549    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-registry-certificates\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.759053 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.759006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-registry-tls\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.759401 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.759378    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-installation-pull-secrets\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.759472 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.759416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-image-registry-private-configuration\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.781747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.781707    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhbmd\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-kube-api-access-xhbmd\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.782694 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.782674    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0ae323bc-9fce-4bf6-9789-7acc02dd1e11-bound-sa-token\") pod \"image-registry-58c7b989d5-9k7cr\" (UID: \"0ae323bc-9fce-4bf6-9789-7acc02dd1e11\") " pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:12.887831 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:12.887795    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:13.186059 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.186029    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-58c7b989d5-9k7cr"]
Jul 21 18:01:13.191660 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:13.191629    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae323bc_9fce_4bf6_9789_7acc02dd1e11.slice/crio-b95cc2338ef1e74046865feffb7939f9283f501324315e25b68a3073bbe3dcd5 WatchSource:0}: Error finding container b95cc2338ef1e74046865feffb7939f9283f501324315e25b68a3073bbe3dcd5: Status 404 returned error can't find the container with id b95cc2338ef1e74046865feffb7939f9283f501324315e25b68a3073bbe3dcd5
Jul 21 18:01:13.194004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.193981    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t"]
Jul 21 18:01:13.196738 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:13.196711    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb76a73fc_f769_45ce_af14_3bd2a0ceea87.slice/crio-52def6607de83265ac7a4471ccb5c3d43b1b2e0d544bbbb0808afc32b268c5b0 WatchSource:0}: Error finding container 52def6607de83265ac7a4471ccb5c3d43b1b2e0d544bbbb0808afc32b268c5b0: Status 404 returned error can't find the container with id 52def6607de83265ac7a4471ccb5c3d43b1b2e0d544bbbb0808afc32b268c5b0
Jul 21 18:01:13.344312 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.344212    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr" event={"ID":"0ae323bc-9fce-4bf6-9789-7acc02dd1e11","Type":"ContainerStarted","Data":"6e526db69d643a9eb914fa5017d3f3de70396dae0ff37f270c7179130c86a3b6"}
Jul 21 18:01:13.344312 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.344270    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr" event={"ID":"0ae323bc-9fce-4bf6-9789-7acc02dd1e11","Type":"ContainerStarted","Data":"b95cc2338ef1e74046865feffb7939f9283f501324315e25b68a3073bbe3dcd5"}
Jul 21 18:01:13.344510 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.344325    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:13.346145 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.346112    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnv29" event={"ID":"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae","Type":"ContainerStarted","Data":"36195150b0f24eb5e3a52cb390cade9001355a436270410b69e3cdfd50429320"}
Jul 21 18:01:13.347231 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.347210    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t" event={"ID":"b76a73fc-f769-45ce-af14-3bd2a0ceea87","Type":"ContainerStarted","Data":"52def6607de83265ac7a4471ccb5c3d43b1b2e0d544bbbb0808afc32b268c5b0"}
Jul 21 18:01:13.391240 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.391180    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr" podStartSLOduration=1.391160822 podStartE2EDuration="1.391160822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:13.390625304 +0000 UTC m=+61.649068484" watchObservedRunningTime="2026-07-21 18:01:13.391160822 +0000 UTC m=+61.649604003"
Jul 21 18:01:13.471317 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:13.471069    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod52da86bd_4022_4f70_b9d9_994fa9aabdf6.slice/crio-3746685759c3992784b706ea1fa60ada4c074d3236c26df64773fdb698b0c6c1 WatchSource:0}: Error finding container 3746685759c3992784b706ea1fa60ada4c074d3236c26df64773fdb698b0c6c1: Status 404 returned error can't find the container with id 3746685759c3992784b706ea1fa60ada4c074d3236c26df64773fdb698b0c6c1
Jul 21 18:01:13.471567 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:13.471541    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc26bca50_ef32_4ecb_8800_f2274a4a484b.slice/crio-40f2d1326c524850b6e721a56be69eda6fd068e96ae3d34e12c1cfb60c64a6ed WatchSource:0}: Error finding container 40f2d1326c524850b6e721a56be69eda6fd068e96ae3d34e12c1cfb60c64a6ed: Status 404 returned error can't find the container with id 40f2d1326c524850b6e721a56be69eda6fd068e96ae3d34e12c1cfb60c64a6ed
Jul 21 18:01:13.471883 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.471856    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"]
Jul 21 18:01:13.473170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:13.473082    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl"]
Jul 21 18:01:14.251786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:14.251756    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7dgs2"
Jul 21 18:01:14.352350 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:14.352313    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf" event={"ID":"c26bca50-ef32-4ecb-8800-f2274a4a484b","Type":"ContainerStarted","Data":"40f2d1326c524850b6e721a56be69eda6fd068e96ae3d34e12c1cfb60c64a6ed"}
Jul 21 18:01:14.353542 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:14.353508    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl" event={"ID":"52da86bd-4022-4f70-b9d9-994fa9aabdf6","Type":"ContainerStarted","Data":"3746685759c3992784b706ea1fa60ada4c074d3236c26df64773fdb698b0c6c1"}
Jul 21 18:01:18.371012 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.370968    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hnv29" event={"ID":"f87c1304-0a50-4bf2-8b8f-c3e8cea362ae","Type":"ContainerStarted","Data":"50b29f104661059fbd7142ce2da54a45000525eb1881aecb369a6e745801b9cb"}
Jul 21 18:01:18.373052 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.373026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t" event={"ID":"b76a73fc-f769-45ce-af14-3bd2a0ceea87","Type":"ContainerStarted","Data":"d9626b80640dfa42dfc8ef4257c4b93784b5ab6c8512b3483c3f36ee3dc7d08b"}
Jul 21 18:01:18.404698 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.404132    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hnv29" podStartSLOduration=17.834057046 podStartE2EDuration="31.404117857s" totalImagesPullingTime="13.570060811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:18.402323714 +0000 UTC m=+66.660766894" watchObservedRunningTime="2026-07-21 18:01:18.404117857 +0000 UTC m=+66.662561036"
Jul 21 18:01:18.914984 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.914943    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:01:18.915165 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.915000    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:01:18.917900 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.917872    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 18:01:18.917900 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.917893    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 18:01:18.928178 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.928130    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/df1e655a-c0cc-4d13-8716-6a0fe968b90d-original-pull-secret\") pod \"global-pull-secret-syncer-zqmhv\" (UID: \"df1e655a-c0cc-4d13-8716-6a0fe968b90d\") " pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:01:18.928178 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:18.928140    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0ef3f2e2-9940-4e62-a294-673fb6176efc-metrics-certs\") pod \"network-metrics-daemon-pkx8c\" (UID: \"0ef3f2e2-9940-4e62-a294-673fb6176efc\") " pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:01:19.015497 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.015456    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:01:19.019646 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.019624    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 18:01:19.019802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.019624    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jg5zx\""
Jul 21 18:01:19.022035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.022018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zqmhv"
Jul 21 18:01:19.026453 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.026435    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-pkx8c"
Jul 21 18:01:19.029421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.029402    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 18:01:19.041195 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.041162    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ddn6\" (UniqueName: \"kubernetes.io/projected/b34e9272-5563-42b1-acce-2e650f738f69-kube-api-access-2ddn6\") pod \"network-check-target-rvdsz\" (UID: \"b34e9272-5563-42b1-acce-2e650f738f69\") " pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:01:19.193237 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.193124    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7f88476b9b-kw99t" podStartSLOduration=2.525755315 podStartE2EDuration="7.193100768s" totalImagesPullingTime="4.667345453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:18.429167876 +0000 UTC m=+66.687611056" watchObservedRunningTime="2026-07-21 18:01:19.193100768 +0000 UTC m=+67.451543973"
Jul 21 18:01:19.193520 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.193501    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-pkx8c"]
Jul 21 18:01:19.196077 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:19.196049    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef3f2e2_9940_4e62_a294_673fb6176efc.slice/crio-a34f9782c5af0c863899efd5152b1378b990c39c280289f898f15d9a3e1010c3 WatchSource:0}: Error finding container a34f9782c5af0c863899efd5152b1378b990c39c280289f898f15d9a3e1010c3: Status 404 returned error can't find the container with id a34f9782c5af0c863899efd5152b1378b990c39c280289f898f15d9a3e1010c3
Jul 21 18:01:19.215994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.215930    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zqmhv"]
Jul 21 18:01:19.219858 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:19.219829    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf1e655a_c0cc_4d13_8716_6a0fe968b90d.slice/crio-e58c218ce0ed60485b4399f0842c746bde90ebcbb02f1ea721ada136b1618e5b WatchSource:0}: Error finding container e58c218ce0ed60485b4399f0842c746bde90ebcbb02f1ea721ada136b1618e5b: Status 404 returned error can't find the container with id e58c218ce0ed60485b4399f0842c746bde90ebcbb02f1ea721ada136b1618e5b
Jul 21 18:01:19.312943 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.312905    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-vznt9\""
Jul 21 18:01:19.320814 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.320658    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:01:19.384070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.383777    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkx8c" event={"ID":"0ef3f2e2-9940-4e62-a294-673fb6176efc","Type":"ContainerStarted","Data":"a34f9782c5af0c863899efd5152b1378b990c39c280289f898f15d9a3e1010c3"}
Jul 21 18:01:19.387907 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.387874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zqmhv" event={"ID":"df1e655a-c0cc-4d13-8716-6a0fe968b90d","Type":"ContainerStarted","Data":"e58c218ce0ed60485b4399f0842c746bde90ebcbb02f1ea721ada136b1618e5b"}
Jul 21 18:01:19.479966 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:19.479932    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rvdsz"]
Jul 21 18:01:19.482546 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:19.482505    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb34e9272_5563_42b1_acce_2e650f738f69.slice/crio-02de08b80b12a47ed89261e85ed5d3c9d05b1abf4acde4757c52afb6cdb9cb3b WatchSource:0}: Error finding container 02de08b80b12a47ed89261e85ed5d3c9d05b1abf4acde4757c52afb6cdb9cb3b: Status 404 returned error can't find the container with id 02de08b80b12a47ed89261e85ed5d3c9d05b1abf4acde4757c52afb6cdb9cb3b
Jul 21 18:01:20.338220 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:20.338175    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cbplt"
Jul 21 18:01:20.394563 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:20.394524    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rvdsz" event={"ID":"b34e9272-5563-42b1-acce-2e650f738f69","Type":"ContainerStarted","Data":"02de08b80b12a47ed89261e85ed5d3c9d05b1abf4acde4757c52afb6cdb9cb3b"}
Jul 21 18:01:22.423674 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:22.423640    2578 patch_prober.go:28] interesting pod/image-registry-5b5f4f5784-9vhpv container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 21 18:01:22.424033 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:22.423693    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv" podUID="a8407270-3386-433d-920b-9266dea58788" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 18:01:23.410603 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:23.410556    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl" event={"ID":"52da86bd-4022-4f70-b9d9-994fa9aabdf6","Type":"ContainerStarted","Data":"4650a57836f6dcc30f17365d4fb42a9182468c83d71ea5bafdd2a23557cfd5a7"}
Jul 21 18:01:23.412250 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:23.412220    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf" event={"ID":"c26bca50-ef32-4ecb-8800-f2274a4a484b","Type":"ContainerStarted","Data":"8ded474df8ea30edc73345706dcfb574ca2fd0d5c6085f42b9ce972d4c12e5ec"}
Jul 21 18:01:23.469317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:23.469270    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf" podStartSLOduration=1.8544421899999999 podStartE2EDuration="11.469255217s" totalImagesPullingTime="9.614813027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:23.468101665 +0000 UTC m=+71.726544847" watchObservedRunningTime="2026-07-21 18:01:23.469255217 +0000 UTC m=+71.727698447"
Jul 21 18:01:24.419855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.419815    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl" event={"ID":"52da86bd-4022-4f70-b9d9-994fa9aabdf6","Type":"ContainerStarted","Data":"8927be9b45df179f76a29150fcd07c480d8c6eb4af544864046fc8c28f5e5a6a"}
Jul 21 18:01:24.420067 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.419875    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl" event={"ID":"52da86bd-4022-4f70-b9d9-994fa9aabdf6","Type":"ContainerStarted","Data":"71361208fd3cbde37a81884689739be176300a7470fd9cbeb8c6c2c30a0ea491"}
Jul 21 18:01:24.423453 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.423298    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkx8c" event={"ID":"0ef3f2e2-9940-4e62-a294-673fb6176efc","Type":"ContainerStarted","Data":"85aa56a106af7bf1a0fccdf6e90fb0d0a85e0d21329b7a6727756b2245a2af5c"}
Jul 21 18:01:24.423453 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.423352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-pkx8c" event={"ID":"0ef3f2e2-9940-4e62-a294-673fb6176efc","Type":"ContainerStarted","Data":"28f526f30fa34770066206eef05d97496f251dc4e9654918ae0f87c22db000a0"}
Jul 21 18:01:24.423453 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.423393    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:24.425278 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.425255    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-99d6b5579-2h5bf"
Jul 21 18:01:24.444689 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.444646    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664b78d767-4kjzl" podStartSLOduration=2.8285897970000002 podStartE2EDuration="12.444630231s" totalImagesPullingTime="9.616040434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:01:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:24.442848166 +0000 UTC m=+72.701291347" watchObservedRunningTime="2026-07-21 18:01:24.444630231 +0000 UTC m=+72.703073411"
Jul 21 18:01:24.485400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:24.485345    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-pkx8c" podStartSLOduration=67.281869918 podStartE2EDuration="1m11.485325431s" totalImagesPullingTime="4.203455513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:24.482750454 +0000 UTC m=+72.741193636" watchObservedRunningTime="2026-07-21 18:01:24.485325431 +0000 UTC m=+72.743768610"
Jul 21 18:01:27.434852 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:27.434814    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zqmhv" event={"ID":"df1e655a-c0cc-4d13-8716-6a0fe968b90d","Type":"ContainerStarted","Data":"6cb394c8b6beba5b9ac49dd78174f3f0b696b1e912e2ba0b740bbf31414ba726"}
Jul 21 18:01:27.436227 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:27.436203    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rvdsz" event={"ID":"b34e9272-5563-42b1-acce-2e650f738f69","Type":"ContainerStarted","Data":"f86e8ebdcc99a89abe69c01e78fdb4b020b33f633336527d11b773307b5d9962"}
Jul 21 18:01:27.436361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:27.436341    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:01:27.462675 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:27.462631    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zqmhv" podStartSLOduration=67.102461524 podStartE2EDuration="1m14.462619231s" totalImagesPullingTime="7.360157707s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:27.461656259 +0000 UTC m=+75.720099440" watchObservedRunningTime="2026-07-21 18:01:27.462619231 +0000 UTC m=+75.721062411"
Jul 21 18:01:27.486467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:27.486421    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rvdsz" podStartSLOduration=67.389330757 podStartE2EDuration="1m14.486406131s" totalImagesPullingTime="7.097075374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:00:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:27.486025001 +0000 UTC m=+75.744468183" watchObservedRunningTime="2026-07-21 18:01:27.486406131 +0000 UTC m=+75.744849324"
Jul 21 18:01:32.422254 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:32.422228    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:01:33.418819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.418786    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-rzcwv"]
Jul 21 18:01:33.421946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.421928    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.424783 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.424760    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 18:01:33.425823 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.425800    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 18:01:33.425823 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.425821    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 18:01:33.426238 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.426221    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 18:01:33.426324 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.426308    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-q5ldh\""
Jul 21 18:01:33.427802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.427783    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 18:01:33.427899 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.427870    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 18:01:33.530099 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxxfc\" (UniqueName: \"kubernetes.io/projected/c022413f-df43-42a6-83db-cda2c9d8536e-kube-api-access-cxxfc\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530099 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530104    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-textfile\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-tls\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530235    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-sys\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530263    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c022413f-df43-42a6-83db-cda2c9d8536e-metrics-client-ca\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530292    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530319    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-root\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530566 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.530566 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.530363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-wtmp\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631120    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-tls\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631174    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-sys\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c022413f-df43-42a6-83db-cda2c9d8536e-metrics-client-ca\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631249    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-root\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631252    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-sys\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631274    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:01:33.631287    2578 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631318    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-wtmp\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:01:33.631355    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-tls podName:c022413f-df43-42a6-83db-cda2c9d8536e nodeName:}" failed. No retries permitted until 2026-07-21 18:01:34.131332169 +0000 UTC m=+82.389775327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-tls") pod "node-exporter-rzcwv" (UID: "c022413f-df43-42a6-83db-cda2c9d8536e") : secret "node-exporter-tls" not found
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631364    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-root\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxxfc\" (UniqueName: \"kubernetes.io/projected/c022413f-df43-42a6-83db-cda2c9d8536e-kube-api-access-cxxfc\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631429    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-wtmp\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631453    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-textfile\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631790    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-textfile\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c022413f-df43-42a6-83db-cda2c9d8536e-metrics-client-ca\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.631989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.631869    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-accelerators-collector-config\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.634026 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.634007    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:33.647206 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:33.647184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxxfc\" (UniqueName: \"kubernetes.io/projected/c022413f-df43-42a6-83db-cda2c9d8536e-kube-api-access-cxxfc\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:34.135932 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:34.135897    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-tls\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:34.138126 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:34.138105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c022413f-df43-42a6-83db-cda2c9d8536e-node-exporter-tls\") pod \"node-exporter-rzcwv\" (UID: \"c022413f-df43-42a6-83db-cda2c9d8536e\") " pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:34.331723 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:34.331690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-rzcwv"
Jul 21 18:01:34.341096 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:01:34.341069    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc022413f_df43_42a6_83db_cda2c9d8536e.slice/crio-64024bc630e178fe709226f82c4291c3830636f65accf07b2081b09a9c5b6c60 WatchSource:0}: Error finding container 64024bc630e178fe709226f82c4291c3830636f65accf07b2081b09a9c5b6c60: Status 404 returned error can't find the container with id 64024bc630e178fe709226f82c4291c3830636f65accf07b2081b09a9c5b6c60
Jul 21 18:01:34.357787 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:34.357767    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-58c7b989d5-9k7cr"
Jul 21 18:01:34.461790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:34.461700    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzcwv" event={"ID":"c022413f-df43-42a6-83db-cda2c9d8536e","Type":"ContainerStarted","Data":"64024bc630e178fe709226f82c4291c3830636f65accf07b2081b09a9c5b6c60"}
Jul 21 18:01:35.467722 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:35.467686    2578 generic.go:356] "Generic (PLEG): container finished" podID="c022413f-df43-42a6-83db-cda2c9d8536e" containerID="ecd381d33bcc3d9386beb891245cb1fc445e3896e687a73697bf8d42c9a1eca7" exitCode=0
Jul 21 18:01:35.468063 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:35.467768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzcwv" event={"ID":"c022413f-df43-42a6-83db-cda2c9d8536e","Type":"ContainerDied","Data":"ecd381d33bcc3d9386beb891245cb1fc445e3896e687a73697bf8d42c9a1eca7"}
Jul 21 18:01:36.474038 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:36.474000    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzcwv" event={"ID":"c022413f-df43-42a6-83db-cda2c9d8536e","Type":"ContainerStarted","Data":"396c9144b23382ac2ea332de0592286cc029167749dff9dada70a871a9f1c806"}
Jul 21 18:01:36.474518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:36.474068    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-rzcwv" event={"ID":"c022413f-df43-42a6-83db-cda2c9d8536e","Type":"ContainerStarted","Data":"9bc64ce2d938cbb63aec4aec98a4e9ec7907d2ece0a9833bb0618de96d44f0c2"}
Jul 21 18:01:36.510782 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:36.510722    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-rzcwv" podStartSLOduration=2.487180766 podStartE2EDuration="3.51070057s" totalImagesPullingTime="959.830612ms" totalInitContainerRuntime="63.689192ms" isStatefulPod=true podCreationTimestamp="2026-07-21 18:01:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:01:36.509231161 +0000 UTC m=+84.767674341" watchObservedRunningTime="2026-07-21 18:01:36.51070057 +0000 UTC m=+84.769143751"
Jul 21 18:01:37.439799 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.439756    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv" podUID="a8407270-3386-433d-920b-9266dea58788" containerName="registry" containerID="cri-o://17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73" gracePeriod=30
Jul 21 18:01:37.678362 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.678339    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:01:37.863107 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863033    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863107 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863066    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-installation-pull-secrets\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863107 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863085    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t4wvk\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-kube-api-access-t4wvk\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863369 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863121    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-trusted-ca\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863369 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863154    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-bound-sa-token\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863369 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8407270-3386-433d-920b-9266dea58788-ca-trust-extracted\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863418    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-registry-certificates\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863486    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-image-registry-private-configuration\") pod \"a8407270-3386-433d-920b-9266dea58788\" (UID: \"a8407270-3386-433d-920b-9266dea58788\") "
Jul 21 18:01:37.863688 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863666    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:01:37.863788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863749    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-trusted-ca\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:37.863913 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.863884    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:01:37.865510 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.865469    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:01:37.865681 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.865643    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-kube-api-access-t4wvk" (OuterVolumeSpecName: "kube-api-access-t4wvk") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "kube-api-access-t4wvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:01:37.865819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.865787    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:01:37.865819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.865811    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:01:37.865955 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.865890    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:01:37.871946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.871924    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a8407270-3386-433d-920b-9266dea58788-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a8407270-3386-433d-920b-9266dea58788" (UID: "a8407270-3386-433d-920b-9266dea58788"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:01:37.964354 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.964329    2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-registry-tls\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:37.964354 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.964351    2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-installation-pull-secrets\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:37.964496 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.964361    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4wvk\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-kube-api-access-t4wvk\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:37.964496 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.964370    2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a8407270-3386-433d-920b-9266dea58788-bound-sa-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:37.964496 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.964379    2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a8407270-3386-433d-920b-9266dea58788-ca-trust-extracted\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:37.964496 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.964408    2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a8407270-3386-433d-920b-9266dea58788-registry-certificates\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:37.964496 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:37.964417    2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a8407270-3386-433d-920b-9266dea58788-image-registry-private-configuration\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:01:38.486953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.486920    2578 generic.go:356] "Generic (PLEG): container finished" podID="a8407270-3386-433d-920b-9266dea58788" containerID="17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73" exitCode=0
Jul 21 18:01:38.487119 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.486975    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv"
Jul 21 18:01:38.487119 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.486998    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv" event={"ID":"a8407270-3386-433d-920b-9266dea58788","Type":"ContainerDied","Data":"17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73"}
Jul 21 18:01:38.487119 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.487057    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b5f4f5784-9vhpv" event={"ID":"a8407270-3386-433d-920b-9266dea58788","Type":"ContainerDied","Data":"147c40256765cde2bc007fc0452b9ceff94f7fc7114e12b3c0621088271cc01b"}
Jul 21 18:01:38.487119 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.487076    2578 scope.go:117] "RemoveContainer" containerID="17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73"
Jul 21 18:01:38.495761 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.495742    2578 scope.go:117] "RemoveContainer" containerID="17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73"
Jul 21 18:01:38.496037 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:01:38.496017    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73\": container with ID starting with 17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73 not found: ID does not exist" containerID="17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73"
Jul 21 18:01:38.496090 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.496045    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73"} err="failed to get container status \"17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73\": rpc error: code = NotFound desc = could not find container \"17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73\": container with ID starting with 17b3161a0a29d1a49faad595d552fae9d495a9661cf7b3f6b128404158a2fd73 not found: ID does not exist"
Jul 21 18:01:38.509135 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.509114    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b5f4f5784-9vhpv"]
Jul 21 18:01:38.515098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:38.515077    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b5f4f5784-9vhpv"]
Jul 21 18:01:39.100574 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:39.100545    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8407270-3386-433d-920b-9266dea58788" path="/var/lib/kubelet/pods/a8407270-3386-433d-920b-9266dea58788/volumes"
Jul 21 18:01:58.442449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:01:58.442413    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rvdsz"
Jul 21 18:05:12.974936 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:12.974906    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:05:12.975529 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:12.975512    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:05:12.981284 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:12.981266    2578 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 18:05:42.211999 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.211959    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"]
Jul 21 18:05:42.213333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.212221    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8407270-3386-433d-920b-9266dea58788" containerName="registry"
Jul 21 18:05:42.213333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.212232    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8407270-3386-433d-920b-9266dea58788" containerName="registry"
Jul 21 18:05:42.213333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.212272    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8407270-3386-433d-920b-9266dea58788" containerName="registry"
Jul 21 18:05:42.213846 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.213831    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.219469 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.219442    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 18:05:42.219586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.219468    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 18:05:42.220135 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.220072    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 18:05:42.220135 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.220070    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 18:05:42.220646 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.220631    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7tzw6\""
Jul 21 18:05:42.236742 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.236722    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"]
Jul 21 18:05:42.263958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.263928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b9640749-756b-4c6c-a645-6ad85dea8231-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.263958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.263961    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9640749-756b-4c6c-a645-6ad85dea8231-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.264141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.263982    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgddh\" (UniqueName: \"kubernetes.io/projected/b9640749-756b-4c6c-a645-6ad85dea8231-kube-api-access-cgddh\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.264141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.264034    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9640749-756b-4c6c-a645-6ad85dea8231-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.264141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.264094    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b9640749-756b-4c6c-a645-6ad85dea8231-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.364764 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.364734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b9640749-756b-4c6c-a645-6ad85dea8231-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.364764 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.364768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9640749-756b-4c6c-a645-6ad85dea8231-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.364958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.364789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgddh\" (UniqueName: \"kubernetes.io/projected/b9640749-756b-4c6c-a645-6ad85dea8231-kube-api-access-cgddh\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.364958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.364806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9640749-756b-4c6c-a645-6ad85dea8231-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.364958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.364824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b9640749-756b-4c6c-a645-6ad85dea8231-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.365277 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.365257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/b9640749-756b-4c6c-a645-6ad85dea8231-manifests\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.367149 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.367115    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9640749-756b-4c6c-a645-6ad85dea8231-tmp\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.367286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.367242    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/b9640749-756b-4c6c-a645-6ad85dea8231-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.367286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.367254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b9640749-756b-4c6c-a645-6ad85dea8231-webhook-cert\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.377361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.377337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgddh\" (UniqueName: \"kubernetes.io/projected/b9640749-756b-4c6c-a645-6ad85dea8231-kube-api-access-cgddh\") pod \"opendatahub-operator-controller-manager-6b54c48d7b-v86px\" (UID: \"b9640749-756b-4c6c-a645-6ad85dea8231\") " pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.523011 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.522928    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:42.668104 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.668054    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"]
Jul 21 18:05:42.670784 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:05:42.670755    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9640749_756b_4c6c_a645_6ad85dea8231.slice/crio-b5f66755cda91f592a1368ef5a57e4b76e97f24ff28b6c5439d83fa4051741f2 WatchSource:0}: Error finding container b5f66755cda91f592a1368ef5a57e4b76e97f24ff28b6c5439d83fa4051741f2: Status 404 returned error can't find the container with id b5f66755cda91f592a1368ef5a57e4b76e97f24ff28b6c5439d83fa4051741f2
Jul 21 18:05:42.672546 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:42.672532    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 18:05:43.268375 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:43.268345    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px" event={"ID":"b9640749-756b-4c6c-a645-6ad85dea8231","Type":"ContainerStarted","Data":"b5f66755cda91f592a1368ef5a57e4b76e97f24ff28b6c5439d83fa4051741f2"}
Jul 21 18:05:46.281724 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:46.281683    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px" event={"ID":"b9640749-756b-4c6c-a645-6ad85dea8231","Type":"ContainerStarted","Data":"4a0d2bbd03a1517047c3d134affaf1b39b42e954ffc2afb42ca9ea8db317f0a1"}
Jul 21 18:05:47.286937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:47.286898    2578 generic.go:356] "Generic (PLEG): container finished" podID="b9640749-756b-4c6c-a645-6ad85dea8231" containerID="4a0d2bbd03a1517047c3d134affaf1b39b42e954ffc2afb42ca9ea8db317f0a1" exitCode=0
Jul 21 18:05:47.287389 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:47.286957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px" event={"ID":"b9640749-756b-4c6c-a645-6ad85dea8231","Type":"ContainerDied","Data":"4a0d2bbd03a1517047c3d134affaf1b39b42e954ffc2afb42ca9ea8db317f0a1"}
Jul 21 18:05:48.292267 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:48.292232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px" event={"ID":"b9640749-756b-4c6c-a645-6ad85dea8231","Type":"ContainerStarted","Data":"ae41301b178e648702d263c9e232e8cf45fe46a1aba8a55a995b212deae82717"}
Jul 21 18:05:48.292710 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:48.292302    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:05:48.327173 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:48.327127    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px" podStartSLOduration=1.803997522 podStartE2EDuration="6.327110986s" totalImagesPullingTime="3.446099106s" totalInitContainerRuntime="1.077014358s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:05:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:05:48.326643053 +0000 UTC m=+336.585086233" watchObservedRunningTime="2026-07-21 18:05:48.327110986 +0000 UTC m=+336.585554166"
Jul 21 18:05:59.299264 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:05:59.299230    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6b54c48d7b-v86px"
Jul 21 18:06:01.672012 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.671975    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"]
Jul 21 18:06:01.681758 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.681738    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.685306 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.685281    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 18:06:01.685628 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.685608    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 18:06:01.685724 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.685608    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 18:06:01.685724 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.685615    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 18:06:01.704317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.704295    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"]
Jul 21 18:06:01.810362 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.810327    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1856fec-efd0-4eae-b789-7675b6c48050-tmp\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.810362 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.810373    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1856fec-efd0-4eae-b789-7675b6c48050-tls-certs\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.810561 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.810479    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5zww\" (UniqueName: \"kubernetes.io/projected/d1856fec-efd0-4eae-b789-7675b6c48050-kube-api-access-w5zww\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.911472 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.911433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5zww\" (UniqueName: \"kubernetes.io/projected/d1856fec-efd0-4eae-b789-7675b6c48050-kube-api-access-w5zww\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.911472 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.911474    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1856fec-efd0-4eae-b789-7675b6c48050-tmp\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.911715 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.911499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1856fec-efd0-4eae-b789-7675b6c48050-tls-certs\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.913706 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.913675    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1856fec-efd0-4eae-b789-7675b6c48050-tmp\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.913882 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.913865    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1856fec-efd0-4eae-b789-7675b6c48050-tls-certs\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.928078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.928017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5zww\" (UniqueName: \"kubernetes.io/projected/d1856fec-efd0-4eae-b789-7675b6c48050-kube-api-access-w5zww\") pod \"kube-auth-proxy-f49f77f5f-7rb7f\" (UID: \"d1856fec-efd0-4eae-b789-7675b6c48050\") " pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:01.951905 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.951878    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bfdvz"]
Jul 21 18:06:01.953890 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.953875    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:01.956769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.956747    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 21 18:06:01.956868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.956769    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-zmmx8\""
Jul 21 18:06:01.968546 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.968525    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bfdvz"]
Jul 21 18:06:01.990412 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:01.990387    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"
Jul 21 18:06:02.012525 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.012500    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:02.012659 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.012535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwkz4\" (UniqueName: \"kubernetes.io/projected/034f126e-24f6-42d4-95c3-5f63faea5d2e-kube-api-access-pwkz4\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:02.113526 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.113496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:02.113526 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.113526    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwkz4\" (UniqueName: \"kubernetes.io/projected/034f126e-24f6-42d4-95c3-5f63faea5d2e-kube-api-access-pwkz4\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:02.113725 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:06:02.113650    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 21 18:06:02.113725 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:06:02.113711    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert podName:034f126e-24f6-42d4-95c3-5f63faea5d2e nodeName:}" failed. No retries permitted until 2026-07-21 18:06:02.613694953 +0000 UTC m=+350.872138110 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert") pod "odh-model-controller-7998c67dc4-bfdvz" (UID: "034f126e-24f6-42d4-95c3-5f63faea5d2e") : secret "odh-model-controller-webhook-cert" not found
Jul 21 18:06:02.137987 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.137961    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f"]
Jul 21 18:06:02.139223 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.139198    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwkz4\" (UniqueName: \"kubernetes.io/projected/034f126e-24f6-42d4-95c3-5f63faea5d2e-kube-api-access-pwkz4\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:02.140354 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:06:02.140329    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1856fec_efd0_4eae_b789_7675b6c48050.slice/crio-9e5a25ce0b09984857e1c6244d1a04703e79758c5860991b230d92091ede121a WatchSource:0}: Error finding container 9e5a25ce0b09984857e1c6244d1a04703e79758c5860991b230d92091ede121a: Status 404 returned error can't find the container with id 9e5a25ce0b09984857e1c6244d1a04703e79758c5860991b230d92091ede121a
Jul 21 18:06:02.341443 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.341366    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f" event={"ID":"d1856fec-efd0-4eae-b789-7675b6c48050","Type":"ContainerStarted","Data":"9e5a25ce0b09984857e1c6244d1a04703e79758c5860991b230d92091ede121a"}
Jul 21 18:06:02.617935 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:02.617847    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:02.618110 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:06:02.618018    2578 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 21 18:06:02.618110 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:06:02.618103    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert podName:034f126e-24f6-42d4-95c3-5f63faea5d2e nodeName:}" failed. No retries permitted until 2026-07-21 18:06:03.618081302 +0000 UTC m=+351.876524460 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert") pod "odh-model-controller-7998c67dc4-bfdvz" (UID: "034f126e-24f6-42d4-95c3-5f63faea5d2e") : secret "odh-model-controller-webhook-cert" not found
Jul 21 18:06:03.107934 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.107896    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5trmw"]
Jul 21 18:06:03.110474 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.110454    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.114377 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.114347    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-7tzxt\""
Jul 21 18:06:03.114507 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.114409    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 21 18:06:03.145554 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.145521    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5trmw"]
Jul 21 18:06:03.222683 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.222642    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xs6l6\" (UniqueName: \"kubernetes.io/projected/d1f86d26-1b3d-4268-bfc8-6223e5167679-kube-api-access-xs6l6\") pod \"model-serving-api-54d89bd5fc-5trmw\" (UID: \"d1f86d26-1b3d-4268-bfc8-6223e5167679\") " pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.222870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.222757    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f86d26-1b3d-4268-bfc8-6223e5167679-tls-certs\") pod \"model-serving-api-54d89bd5fc-5trmw\" (UID: \"d1f86d26-1b3d-4268-bfc8-6223e5167679\") " pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.323445 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.323414    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xs6l6\" (UniqueName: \"kubernetes.io/projected/d1f86d26-1b3d-4268-bfc8-6223e5167679-kube-api-access-xs6l6\") pod \"model-serving-api-54d89bd5fc-5trmw\" (UID: \"d1f86d26-1b3d-4268-bfc8-6223e5167679\") " pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.323642 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.323467    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f86d26-1b3d-4268-bfc8-6223e5167679-tls-certs\") pod \"model-serving-api-54d89bd5fc-5trmw\" (UID: \"d1f86d26-1b3d-4268-bfc8-6223e5167679\") " pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.323642 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:06:03.323568    2578 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 21 18:06:03.323766 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:06:03.323653    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1f86d26-1b3d-4268-bfc8-6223e5167679-tls-certs podName:d1f86d26-1b3d-4268-bfc8-6223e5167679 nodeName:}" failed. No retries permitted until 2026-07-21 18:06:03.8236357 +0000 UTC m=+352.082078879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/d1f86d26-1b3d-4268-bfc8-6223e5167679-tls-certs") pod "model-serving-api-54d89bd5fc-5trmw" (UID: "d1f86d26-1b3d-4268-bfc8-6223e5167679") : secret "model-serving-api-tls" not found
Jul 21 18:06:03.341501 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.341474    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xs6l6\" (UniqueName: \"kubernetes.io/projected/d1f86d26-1b3d-4268-bfc8-6223e5167679-kube-api-access-xs6l6\") pod \"model-serving-api-54d89bd5fc-5trmw\" (UID: \"d1f86d26-1b3d-4268-bfc8-6223e5167679\") " pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.626759 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.626685    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:03.629020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.628992    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/034f126e-24f6-42d4-95c3-5f63faea5d2e-cert\") pod \"odh-model-controller-7998c67dc4-bfdvz\" (UID: \"034f126e-24f6-42d4-95c3-5f63faea5d2e\") " pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:03.763082 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.763050    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:03.778799 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.778767    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"]
Jul 21 18:06:03.781254 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.781230    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.784629 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.784550    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 18:06:03.784629 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.784606    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 18:06:03.784939 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.784873    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 18:06:03.785121 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.785099    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-76dl7\""
Jul 21 18:06:03.785252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.785124    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 18:06:03.828053 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828020    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-token\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.828172 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828130    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.828213 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.828213 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828201    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.828291 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828223    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-cacerts\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.828291 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f86d26-1b3d-4268-bfc8-6223e5167679-tls-certs\") pod \"model-serving-api-54d89bd5fc-5trmw\" (UID: \"d1f86d26-1b3d-4268-bfc8-6223e5167679\") " pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.828291 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828285    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/12197e06-305b-4513-8f3d-49857e8cfbbf-local-certs\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.828388 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.828312    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc5lf\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-kube-api-access-tc5lf\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.830729 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.830704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d1f86d26-1b3d-4268-bfc8-6223e5167679-tls-certs\") pod \"model-serving-api-54d89bd5fc-5trmw\" (UID: \"d1f86d26-1b3d-4268-bfc8-6223e5167679\") " pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:03.852747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.852721    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"]
Jul 21 18:06:03.900580 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.900554    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-bfdvz"]
Jul 21 18:06:03.903096 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:06:03.903076    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod034f126e_24f6_42d4_95c3_5f63faea5d2e.slice/crio-2929ce74973eea37770c51e7de2adb39696eee72a1ad4573dce7ac945ef1d6df WatchSource:0}: Error finding container 2929ce74973eea37770c51e7de2adb39696eee72a1ad4573dce7ac945ef1d6df: Status 404 returned error can't find the container with id 2929ce74973eea37770c51e7de2adb39696eee72a1ad4573dce7ac945ef1d6df
Jul 21 18:06:03.929393 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.929368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.929481 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.929396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-cacerts\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.929481 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.929420    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/12197e06-305b-4513-8f3d-49857e8cfbbf-local-certs\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.929481 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.929439    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc5lf\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-kube-api-access-tc5lf\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.929481 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.929471    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-token\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.929696 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.929541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.929696 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.929604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.930052 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.930030    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.931655 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.931637    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/12197e06-305b-4513-8f3d-49857e8cfbbf-local-certs\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.931862 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.931842    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-cacerts\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.931904 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.931892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.931938 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.931891    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.938845 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.938817    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc5lf\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-kube-api-access-tc5lf\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:03.940777 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:03.940753    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-token\") pod \"istiod-openshift-gateway-7574459656-fmvdm\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:04.022077 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:04.022035    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:04.093107 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:04.093078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:04.188702 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:04.188647    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-5trmw"]
Jul 21 18:06:04.193481 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:06:04.193441    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1f86d26_1b3d_4268_bfc8_6223e5167679.slice/crio-d3a635227ad80ae2ef2aa9ad8c6bcbfe3efca055fd8bc5e4a823a3565221d6f3 WatchSource:0}: Error finding container d3a635227ad80ae2ef2aa9ad8c6bcbfe3efca055fd8bc5e4a823a3565221d6f3: Status 404 returned error can't find the container with id d3a635227ad80ae2ef2aa9ad8c6bcbfe3efca055fd8bc5e4a823a3565221d6f3
Jul 21 18:06:04.257554 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:06:04.257488    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12197e06_305b_4513_8f3d_49857e8cfbbf.slice/crio-198958158af532aacf4baf5ec6040a6391cd91f40492944f087537f33a0aba2c WatchSource:0}: Error finding container 198958158af532aacf4baf5ec6040a6391cd91f40492944f087537f33a0aba2c: Status 404 returned error can't find the container with id 198958158af532aacf4baf5ec6040a6391cd91f40492944f087537f33a0aba2c
Jul 21 18:06:04.257890 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:04.257860    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"]
Jul 21 18:06:04.350955 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:04.350918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" event={"ID":"12197e06-305b-4513-8f3d-49857e8cfbbf","Type":"ContainerStarted","Data":"198958158af532aacf4baf5ec6040a6391cd91f40492944f087537f33a0aba2c"}
Jul 21 18:06:04.352292 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:04.352262    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5trmw" event={"ID":"d1f86d26-1b3d-4268-bfc8-6223e5167679","Type":"ContainerStarted","Data":"d3a635227ad80ae2ef2aa9ad8c6bcbfe3efca055fd8bc5e4a823a3565221d6f3"}
Jul 21 18:06:04.353403 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:04.353377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz" event={"ID":"034f126e-24f6-42d4-95c3-5f63faea5d2e","Type":"ContainerStarted","Data":"2929ce74973eea37770c51e7de2adb39696eee72a1ad4573dce7ac945ef1d6df"}
Jul 21 18:06:05.363703 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:05.363148    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f" event={"ID":"d1856fec-efd0-4eae-b789-7675b6c48050","Type":"ContainerStarted","Data":"8361bac94e7024324f0eab73704d597227dc91173a1a568188c9008ce1236f57"}
Jul 21 18:06:05.385633 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:05.385015    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f49f77f5f-7rb7f" podStartSLOduration=1.540272155 podStartE2EDuration="4.384995032s" totalImagesPullingTime="2.844722877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:05.38391695 +0000 UTC m=+353.642360131" watchObservedRunningTime="2026-07-21 18:06:05.384995032 +0000 UTC m=+353.643438214"
Jul 21 18:06:08.553031 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:08.552996    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:06:08.553363 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:08.553063    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:06:09.380582 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.380545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" event={"ID":"12197e06-305b-4513-8f3d-49857e8cfbbf","Type":"ContainerStarted","Data":"e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba"}
Jul 21 18:06:09.380810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.380740    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:09.382330 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.382297    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-5trmw" event={"ID":"d1f86d26-1b3d-4268-bfc8-6223e5167679","Type":"ContainerStarted","Data":"30a6a82a6255745875ae2cf29ddf12652b032cd7db017ce6576f5c40ae09c82e"}
Jul 21 18:06:09.382483 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.382380    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:09.382685 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.382660    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-fmvdm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 18:06:09.382810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.382709    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" podUID="12197e06-305b-4513-8f3d-49857e8cfbbf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 18:06:09.383839 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.383819    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz" event={"ID":"034f126e-24f6-42d4-95c3-5f63faea5d2e","Type":"ContainerStarted","Data":"ca839652089a2f48a342d468727dc9689d6166e43d5c0151be16f8cbede74280"}
Jul 21 18:06:09.383963 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.383877    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:09.456958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.456906    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" podStartSLOduration=2.163878483 podStartE2EDuration="6.456887926s" totalImagesPullingTime="4.293009443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:09.456569497 +0000 UTC m=+357.715012677" watchObservedRunningTime="2026-07-21 18:06:09.456887926 +0000 UTC m=+357.715331107"
Jul 21 18:06:09.511648 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:09.511568    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz" podStartSLOduration=3.8559301870000002 podStartE2EDuration="8.511547226s" totalImagesPullingTime="4.655617039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:09.503815276 +0000 UTC m=+357.762258455" watchObservedRunningTime="2026-07-21 18:06:09.511547226 +0000 UTC m=+357.769990406"
Jul 21 18:06:10.388347 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.388307    2578 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-fmvdm container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 18:06:10.388797 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.388374    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" podUID="12197e06-305b-4513-8f3d-49857e8cfbbf" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 18:06:10.582958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.582832    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-5trmw" podStartSLOduration=3.228684892 podStartE2EDuration="7.582811215s" totalImagesPullingTime="4.354126323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:09.543815014 +0000 UTC m=+357.802258196" watchObservedRunningTime="2026-07-21 18:06:10.582811215 +0000 UTC m=+358.841254397"
Jul 21 18:06:10.584145 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.584124    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"]
Jul 21 18:06:10.587397 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.587381    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.605010 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.604989    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-skrlf\""
Jul 21 18:06:10.638039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.638011    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"]
Jul 21 18:06:10.693935 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.693868    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.693935 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.693904    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.693935 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.693923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.694106 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.693970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.694106 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.694030    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.694106 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.694059    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.694106 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.694083    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjrg4\" (UniqueName: \"kubernetes.io/projected/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-kube-api-access-fjrg4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.694221 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.694119    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.694221 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.694145    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.794815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.794780    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.794960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.794825    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.794960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.794873    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.794960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.794895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.794960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.794913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.794960    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.795001    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.795035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjrg4\" (UniqueName: \"kubernetes.io/projected/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-kube-api-access-fjrg4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.795062    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795482 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.795457    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.795524    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795671 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.795575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.795763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.795662    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.796149 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.796129    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.797465 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.797436    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.797664 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.797644    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.816793 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.816767    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.819675 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.819654    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjrg4\" (UniqueName: \"kubernetes.io/projected/44bffe7e-2c75-4c93-98c7-1cca9b6818b7-kube-api-access-fjrg4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47227g\" (UID: \"44bffe7e-2c75-4c93-98c7-1cca9b6818b7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:10.897445 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:10.897413    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:11.052824 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:11.052792    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"]
Jul 21 18:06:11.057126 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:06:11.057103    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44bffe7e_2c75_4c93_98c7_1cca9b6818b7.slice/crio-a8788575d1e035938bbb66c58bd18627a5677fb7f546dad457f8f95039a8c86b WatchSource:0}: Error finding container a8788575d1e035938bbb66c58bd18627a5677fb7f546dad457f8f95039a8c86b: Status 404 returned error can't find the container with id a8788575d1e035938bbb66c58bd18627a5677fb7f546dad457f8f95039a8c86b
Jul 21 18:06:11.392568 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:11.392531    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g" event={"ID":"44bffe7e-2c75-4c93-98c7-1cca9b6818b7","Type":"ContainerStarted","Data":"a8788575d1e035938bbb66c58bd18627a5677fb7f546dad457f8f95039a8c86b"}
Jul 21 18:06:13.389239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.389151    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:13.719735 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.719650    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"]
Jul 21 18:06:13.728123 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.728100    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:13.729017 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.728996    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"]
Jul 21 18:06:13.731747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.731724    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-p7h75\""
Jul 21 18:06:13.732162 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.732143    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:13.737485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.737465    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 21 18:06:13.749159 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.749136    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"]
Jul 21 18:06:13.751385 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.751361    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mhx27\""
Jul 21 18:06:13.762792 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.762764    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"]
Jul 21 18:06:13.764378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.764357    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 21 18:06:13.822036 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.822000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v22kr\" (UniqueName: \"kubernetes.io/projected/b85bf252-4660-4a31-beda-038aab8cabc4-kube-api-access-v22kr\") pod \"llmisvc-controller-manager-777c6bd7d5-cs9n8\" (UID: \"b85bf252-4660-4a31-beda-038aab8cabc4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:13.822196 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.822039    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23a33cf6-c769-4bb9-8a5f-ce716d059ed8-cert\") pod \"kserve-controller-manager-d6f4b65b8-x87b7\" (UID: \"23a33cf6-c769-4bb9-8a5f-ce716d059ed8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:13.822196 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.822081    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftcpq\" (UniqueName: \"kubernetes.io/projected/23a33cf6-c769-4bb9-8a5f-ce716d059ed8-kube-api-access-ftcpq\") pod \"kserve-controller-manager-d6f4b65b8-x87b7\" (UID: \"23a33cf6-c769-4bb9-8a5f-ce716d059ed8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:13.822196 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.822122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b85bf252-4660-4a31-beda-038aab8cabc4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cs9n8\" (UID: \"b85bf252-4660-4a31-beda-038aab8cabc4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:13.922681 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.922644    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v22kr\" (UniqueName: \"kubernetes.io/projected/b85bf252-4660-4a31-beda-038aab8cabc4-kube-api-access-v22kr\") pod \"llmisvc-controller-manager-777c6bd7d5-cs9n8\" (UID: \"b85bf252-4660-4a31-beda-038aab8cabc4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:13.922857 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.922707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23a33cf6-c769-4bb9-8a5f-ce716d059ed8-cert\") pod \"kserve-controller-manager-d6f4b65b8-x87b7\" (UID: \"23a33cf6-c769-4bb9-8a5f-ce716d059ed8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:13.922857 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.922752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftcpq\" (UniqueName: \"kubernetes.io/projected/23a33cf6-c769-4bb9-8a5f-ce716d059ed8-kube-api-access-ftcpq\") pod \"kserve-controller-manager-d6f4b65b8-x87b7\" (UID: \"23a33cf6-c769-4bb9-8a5f-ce716d059ed8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:13.922857 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.922785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b85bf252-4660-4a31-beda-038aab8cabc4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cs9n8\" (UID: \"b85bf252-4660-4a31-beda-038aab8cabc4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:13.925233 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.925201    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b85bf252-4660-4a31-beda-038aab8cabc4-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-cs9n8\" (UID: \"b85bf252-4660-4a31-beda-038aab8cabc4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:13.925726 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.925705    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/23a33cf6-c769-4bb9-8a5f-ce716d059ed8-cert\") pod \"kserve-controller-manager-d6f4b65b8-x87b7\" (UID: \"23a33cf6-c769-4bb9-8a5f-ce716d059ed8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:13.935233 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.935197    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v22kr\" (UniqueName: \"kubernetes.io/projected/b85bf252-4660-4a31-beda-038aab8cabc4-kube-api-access-v22kr\") pod \"llmisvc-controller-manager-777c6bd7d5-cs9n8\" (UID: \"b85bf252-4660-4a31-beda-038aab8cabc4\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:13.937182 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:13.937160    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftcpq\" (UniqueName: \"kubernetes.io/projected/23a33cf6-c769-4bb9-8a5f-ce716d059ed8-kube-api-access-ftcpq\") pod \"kserve-controller-manager-d6f4b65b8-x87b7\" (UID: \"23a33cf6-c769-4bb9-8a5f-ce716d059ed8\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:14.010560 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.010521    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:06:14.010721 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.010637    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:06:14.010721 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.010670    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:06:14.041378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.041354    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:14.048851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.048830    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:14.104518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.104180    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"]
Jul 21 18:06:14.104518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.104401    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" podUID="12197e06-305b-4513-8f3d-49857e8cfbbf" containerName="discovery" containerID="cri-o://e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba" gracePeriod=30
Jul 21 18:06:14.283531 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.283447    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"]
Jul 21 18:06:14.287507 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:06:14.287474    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb85bf252_4660_4a31_beda_038aab8cabc4.slice/crio-edab92eb690607d442c5dd1fa7f61190a980e8fd2451dff7417b92c62ebac955 WatchSource:0}: Error finding container edab92eb690607d442c5dd1fa7f61190a980e8fd2451dff7417b92c62ebac955: Status 404 returned error can't find the container with id edab92eb690607d442c5dd1fa7f61190a980e8fd2451dff7417b92c62ebac955
Jul 21 18:06:14.302219 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.302138    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"]
Jul 21 18:06:14.305146 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:06:14.305106    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23a33cf6_c769_4bb9_8a5f_ce716d059ed8.slice/crio-713189fa5f3da4ab6fb1ddf7922580ac4748f5f4ed985f046754aeadffb64090 WatchSource:0}: Error finding container 713189fa5f3da4ab6fb1ddf7922580ac4748f5f4ed985f046754aeadffb64090: Status 404 returned error can't find the container with id 713189fa5f3da4ab6fb1ddf7922580ac4748f5f4ed985f046754aeadffb64090
Jul 21 18:06:14.355582 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.355551    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:14.404976 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.404945    2578 generic.go:356] "Generic (PLEG): container finished" podID="12197e06-305b-4513-8f3d-49857e8cfbbf" containerID="e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba" exitCode=0
Jul 21 18:06:14.405415 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.404984    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"
Jul 21 18:06:14.405415 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.405029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" event={"ID":"12197e06-305b-4513-8f3d-49857e8cfbbf","Type":"ContainerDied","Data":"e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba"}
Jul 21 18:06:14.405415 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.405094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm" event={"ID":"12197e06-305b-4513-8f3d-49857e8cfbbf","Type":"ContainerDied","Data":"198958158af532aacf4baf5ec6040a6391cd91f40492944f087537f33a0aba2c"}
Jul 21 18:06:14.405415 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.405127    2578 scope.go:117] "RemoveContainer" containerID="e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba"
Jul 21 18:06:14.406639 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.406618    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g" event={"ID":"44bffe7e-2c75-4c93-98c7-1cca9b6818b7","Type":"ContainerStarted","Data":"3ca0f42c84959f1df1a8dd4f140e3faca3ae6a67049deab8ba9496f663c5df31"}
Jul 21 18:06:14.407725 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.407704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8" event={"ID":"b85bf252-4660-4a31-beda-038aab8cabc4","Type":"ContainerStarted","Data":"edab92eb690607d442c5dd1fa7f61190a980e8fd2451dff7417b92c62ebac955"}
Jul 21 18:06:14.408855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.408839    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7" event={"ID":"23a33cf6-c769-4bb9-8a5f-ce716d059ed8","Type":"ContainerStarted","Data":"713189fa5f3da4ab6fb1ddf7922580ac4748f5f4ed985f046754aeadffb64090"}
Jul 21 18:06:14.413070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.413055    2578 scope.go:117] "RemoveContainer" containerID="e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba"
Jul 21 18:06:14.413292 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:06:14.413275    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba\": container with ID starting with e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba not found: ID does not exist" containerID="e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba"
Jul 21 18:06:14.413345 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.413298    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba"} err="failed to get container status \"e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba\": rpc error: code = NotFound desc = could not find container \"e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba\": container with ID starting with e9d8fce3577520cc5ee52760a7b5efba0e4037b4902b6ac51c8ac2bcc05855ba not found: ID does not exist"
Jul 21 18:06:14.430294 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.430276    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tc5lf\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-kube-api-access-tc5lf\") pod \"12197e06-305b-4513-8f3d-49857e8cfbbf\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") "
Jul 21 18:06:14.430383 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.430330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-kubeconfig\") pod \"12197e06-305b-4513-8f3d-49857e8cfbbf\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") "
Jul 21 18:06:14.430383 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.430356    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-token\") pod \"12197e06-305b-4513-8f3d-49857e8cfbbf\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") "
Jul 21 18:06:14.430473 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.430457    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/12197e06-305b-4513-8f3d-49857e8cfbbf-local-certs\") pod \"12197e06-305b-4513-8f3d-49857e8cfbbf\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") "
Jul 21 18:06:14.430512 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.430500    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-dns-cert\") pod \"12197e06-305b-4513-8f3d-49857e8cfbbf\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") "
Jul 21 18:06:14.430552 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.430529    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-ca-configmap\") pod \"12197e06-305b-4513-8f3d-49857e8cfbbf\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") "
Jul 21 18:06:14.430552 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.430548    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-cacerts\") pod \"12197e06-305b-4513-8f3d-49857e8cfbbf\" (UID: \"12197e06-305b-4513-8f3d-49857e8cfbbf\") "
Jul 21 18:06:14.431425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.431357    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "12197e06-305b-4513-8f3d-49857e8cfbbf" (UID: "12197e06-305b-4513-8f3d-49857e8cfbbf"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:06:14.433577 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.433550    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-kube-api-access-tc5lf" (OuterVolumeSpecName: "kube-api-access-tc5lf") pod "12197e06-305b-4513-8f3d-49857e8cfbbf" (UID: "12197e06-305b-4513-8f3d-49857e8cfbbf"). InnerVolumeSpecName "kube-api-access-tc5lf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:06:14.433780 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.433759    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12197e06-305b-4513-8f3d-49857e8cfbbf-local-certs" (OuterVolumeSpecName: "local-certs") pod "12197e06-305b-4513-8f3d-49857e8cfbbf" (UID: "12197e06-305b-4513-8f3d-49857e8cfbbf"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:06:14.433895 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.433794    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-token" (OuterVolumeSpecName: "istio-token") pod "12197e06-305b-4513-8f3d-49857e8cfbbf" (UID: "12197e06-305b-4513-8f3d-49857e8cfbbf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:06:14.433895 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.433881    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "12197e06-305b-4513-8f3d-49857e8cfbbf" (UID: "12197e06-305b-4513-8f3d-49857e8cfbbf"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:06:14.434004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.433911    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "12197e06-305b-4513-8f3d-49857e8cfbbf" (UID: "12197e06-305b-4513-8f3d-49857e8cfbbf"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:06:14.434324 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.434296    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-cacerts" (OuterVolumeSpecName: "cacerts") pod "12197e06-305b-4513-8f3d-49857e8cfbbf" (UID: "12197e06-305b-4513-8f3d-49857e8cfbbf"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:06:14.505413 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.505366    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g" podStartSLOduration=1.554265472 podStartE2EDuration="4.505351252s" totalImagesPullingTime="2.95108578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:14.505113406 +0000 UTC m=+362.763556596" watchObservedRunningTime="2026-07-21 18:06:14.505351252 +0000 UTC m=+362.763794431"
Jul 21 18:06:14.531777 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.531743    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-kubeconfig\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:06:14.531777 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.531775    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:06:14.531937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.531789    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/12197e06-305b-4513-8f3d-49857e8cfbbf-local-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:06:14.531937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.531801    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-dns-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:06:14.531937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.531814    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/12197e06-305b-4513-8f3d-49857e8cfbbf-istio-csr-ca-configmap\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:06:14.531937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.531830    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/12197e06-305b-4513-8f3d-49857e8cfbbf-cacerts\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:06:14.531937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.531846    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tc5lf\" (UniqueName: \"kubernetes.io/projected/12197e06-305b-4513-8f3d-49857e8cfbbf-kube-api-access-tc5lf\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:06:14.731082 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.731049    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"]
Jul 21 18:06:14.737353 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.737330    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-fmvdm"]
Jul 21 18:06:14.898335 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:14.898297    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:15.102455 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:15.102389    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12197e06-305b-4513-8f3d-49857e8cfbbf" path="/var/lib/kubelet/pods/12197e06-305b-4513-8f3d-49857e8cfbbf/volumes"
Jul 21 18:06:15.903346 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:15.903318    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:16.419827 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:16.419790    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:16.421003 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:16.420977    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47227g"
Jul 21 18:06:17.425948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:17.425904    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7" event={"ID":"23a33cf6-c769-4bb9-8a5f-ce716d059ed8","Type":"ContainerStarted","Data":"b4611fe2ab370a6827b4da91cb5a9265e5132de72af5525ece651f32c082615d"}
Jul 21 18:06:17.426430 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:17.426064    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:17.479256 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:17.479201    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7" podStartSLOduration=2.010682108 podStartE2EDuration="4.4791825s" totalImagesPullingTime="2.468500392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:17.477649842 +0000 UTC m=+365.736093026" watchObservedRunningTime="2026-07-21 18:06:17.4791825 +0000 UTC m=+365.737625659"
Jul 21 18:06:18.431316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:18.431280    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8" event={"ID":"b85bf252-4660-4a31-beda-038aab8cabc4","Type":"ContainerStarted","Data":"d158db1e4bde39545d8ed6c205aa0f56cd8c0b20d9ca6fddc4bf680933c8cdd3"}
Jul 21 18:06:18.460742 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:18.460697    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8" podStartSLOduration=2.13561582 podStartE2EDuration="5.460682018s" totalImagesPullingTime="3.325066198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:06:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:06:18.459782593 +0000 UTC m=+366.718225797" watchObservedRunningTime="2026-07-21 18:06:18.460682018 +0000 UTC m=+366.719125277"
Jul 21 18:06:19.440338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:19.440305    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:06:20.390005 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:20.389972    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-bfdvz"
Jul 21 18:06:20.391877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:20.391846    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-5trmw"
Jul 21 18:06:48.436861 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:48.436833    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x87b7"
Jul 21 18:06:50.446562 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:06:50.446531    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-cs9n8"
Jul 21 18:07:02.619926 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.619891    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mwblh"]
Jul 21 18:07:02.620359 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.620173    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12197e06-305b-4513-8f3d-49857e8cfbbf" containerName="discovery"
Jul 21 18:07:02.620359 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.620183    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="12197e06-305b-4513-8f3d-49857e8cfbbf" containerName="discovery"
Jul 21 18:07:02.620359 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.620224    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="12197e06-305b-4513-8f3d-49857e8cfbbf" containerName="discovery"
Jul 21 18:07:02.631435 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.631411    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mwblh"
Jul 21 18:07:02.636637 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.636616    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 18:07:02.636779 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.636673    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 18:07:02.638077 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.638056    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-jxv6t\""
Jul 21 18:07:02.644449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.644425    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mwblh"]
Jul 21 18:07:02.724218 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.724185    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p967\" (UniqueName: \"kubernetes.io/projected/0d0cef02-335f-4858-b80e-310bf99d4e73-kube-api-access-4p967\") pod \"kuadrant-operator-catalog-mwblh\" (UID: \"0d0cef02-335f-4858-b80e-310bf99d4e73\") " pod="kuadrant-system/kuadrant-operator-catalog-mwblh"
Jul 21 18:07:02.825555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.825519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p967\" (UniqueName: \"kubernetes.io/projected/0d0cef02-335f-4858-b80e-310bf99d4e73-kube-api-access-4p967\") pod \"kuadrant-operator-catalog-mwblh\" (UID: \"0d0cef02-335f-4858-b80e-310bf99d4e73\") " pod="kuadrant-system/kuadrant-operator-catalog-mwblh"
Jul 21 18:07:02.834671 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.834641    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p967\" (UniqueName: \"kubernetes.io/projected/0d0cef02-335f-4858-b80e-310bf99d4e73-kube-api-access-4p967\") pod \"kuadrant-operator-catalog-mwblh\" (UID: \"0d0cef02-335f-4858-b80e-310bf99d4e73\") " pod="kuadrant-system/kuadrant-operator-catalog-mwblh"
Jul 21 18:07:02.942500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.942467    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mwblh"
Jul 21 18:07:02.990948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:02.990917    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mwblh"]
Jul 21 18:07:03.086462 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.086437    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mwblh"]
Jul 21 18:07:03.089130 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:07:03.089096    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d0cef02_335f_4858_b80e_310bf99d4e73.slice/crio-e734fcdeb837c0c3e212fb43ffe949d06f6c0fe378b4e6e01da16ba16c3816f8 WatchSource:0}: Error finding container e734fcdeb837c0c3e212fb43ffe949d06f6c0fe378b4e6e01da16ba16c3816f8: Status 404 returned error can't find the container with id e734fcdeb837c0c3e212fb43ffe949d06f6c0fe378b4e6e01da16ba16c3816f8
Jul 21 18:07:03.177098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.177054    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-754jw"]
Jul 21 18:07:03.180979 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.180955    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:03.187498 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.187469    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-754jw"]
Jul 21 18:07:03.329642 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.329517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l78h\" (UniqueName: \"kubernetes.io/projected/f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7-kube-api-access-7l78h\") pod \"kuadrant-operator-catalog-754jw\" (UID: \"f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7\") " pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:03.429986 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.429947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l78h\" (UniqueName: \"kubernetes.io/projected/f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7-kube-api-access-7l78h\") pod \"kuadrant-operator-catalog-754jw\" (UID: \"f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7\") " pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:03.438888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.438857    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l78h\" (UniqueName: \"kubernetes.io/projected/f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7-kube-api-access-7l78h\") pod \"kuadrant-operator-catalog-754jw\" (UID: \"f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7\") " pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:03.491252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.491210    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:03.611344 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.611272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mwblh" event={"ID":"0d0cef02-335f-4858-b80e-310bf99d4e73","Type":"ContainerStarted","Data":"e734fcdeb837c0c3e212fb43ffe949d06f6c0fe378b4e6e01da16ba16c3816f8"}
Jul 21 18:07:03.623815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:03.623796    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-754jw"]
Jul 21 18:07:03.626227 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:07:03.626204    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf82fde1e_10eb_4ed3_89f8_dfb152d6e4e7.slice/crio-49582f2b388de3770f75e0bf60fec30c05b15c8debe1d8d659939f7e143f663c WatchSource:0}: Error finding container 49582f2b388de3770f75e0bf60fec30c05b15c8debe1d8d659939f7e143f663c: Status 404 returned error can't find the container with id 49582f2b388de3770f75e0bf60fec30c05b15c8debe1d8d659939f7e143f663c
Jul 21 18:07:04.617513 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:04.617477    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-754jw" event={"ID":"f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7","Type":"ContainerStarted","Data":"49582f2b388de3770f75e0bf60fec30c05b15c8debe1d8d659939f7e143f663c"}
Jul 21 18:07:06.628244 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.628209    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-754jw" event={"ID":"f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7","Type":"ContainerStarted","Data":"51bb4a245dbd0aef13c4280cba8db7f3e4dff1d69dad1382cb5e6efef5ab1e8f"}
Jul 21 18:07:06.629682 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.629656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mwblh" event={"ID":"0d0cef02-335f-4858-b80e-310bf99d4e73","Type":"ContainerStarted","Data":"a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f"}
Jul 21 18:07:06.629786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.629721    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-mwblh" podUID="0d0cef02-335f-4858-b80e-310bf99d4e73" containerName="registry-server" containerID="cri-o://a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f" gracePeriod=2
Jul 21 18:07:06.647775 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.647735    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-754jw" podStartSLOduration=1.232731258 podStartE2EDuration="3.647721157s" totalImagesPullingTime="2.414989899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:07:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:07:06.646971588 +0000 UTC m=+414.905414764" watchObservedRunningTime="2026-07-21 18:07:06.647721157 +0000 UTC m=+414.906164336"
Jul 21 18:07:06.668506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.668469    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-mwblh" podStartSLOduration=1.715862937 podStartE2EDuration="4.668455279s" totalImagesPullingTime="2.952592342s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:07:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:07:06.667482744 +0000 UTC m=+414.925925924" watchObservedRunningTime="2026-07-21 18:07:06.668455279 +0000 UTC m=+414.926898460"
Jul 21 18:07:06.866571 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.866549    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mwblh"
Jul 21 18:07:06.961055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.961024    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p967\" (UniqueName: \"kubernetes.io/projected/0d0cef02-335f-4858-b80e-310bf99d4e73-kube-api-access-4p967\") pod \"0d0cef02-335f-4858-b80e-310bf99d4e73\" (UID: \"0d0cef02-335f-4858-b80e-310bf99d4e73\") "
Jul 21 18:07:06.963110 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:06.963087    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d0cef02-335f-4858-b80e-310bf99d4e73-kube-api-access-4p967" (OuterVolumeSpecName: "kube-api-access-4p967") pod "0d0cef02-335f-4858-b80e-310bf99d4e73" (UID: "0d0cef02-335f-4858-b80e-310bf99d4e73"). InnerVolumeSpecName "kube-api-access-4p967". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:07:07.062548 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.062215    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p967\" (UniqueName: \"kubernetes.io/projected/0d0cef02-335f-4858-b80e-310bf99d4e73-kube-api-access-4p967\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:07:07.634148 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.634111    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d0cef02-335f-4858-b80e-310bf99d4e73" containerID="a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f" exitCode=0
Jul 21 18:07:07.634631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.634155    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mwblh"
Jul 21 18:07:07.634631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.634196    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mwblh" event={"ID":"0d0cef02-335f-4858-b80e-310bf99d4e73","Type":"ContainerDied","Data":"a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f"}
Jul 21 18:07:07.634631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.634260    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mwblh" event={"ID":"0d0cef02-335f-4858-b80e-310bf99d4e73","Type":"ContainerDied","Data":"e734fcdeb837c0c3e212fb43ffe949d06f6c0fe378b4e6e01da16ba16c3816f8"}
Jul 21 18:07:07.634631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.634301    2578 scope.go:117] "RemoveContainer" containerID="a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f"
Jul 21 18:07:07.642865 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.642847    2578 scope.go:117] "RemoveContainer" containerID="a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f"
Jul 21 18:07:07.643101 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:07:07.643085    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f\": container with ID starting with a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f not found: ID does not exist" containerID="a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f"
Jul 21 18:07:07.643155 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.643110    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f"} err="failed to get container status \"a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f\": rpc error: code = NotFound desc = could not find container \"a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f\": container with ID starting with a871721dc3e5a54e14c63946ed0866fa7e7b245ecc76e81ffa6a26687670955f not found: ID does not exist"
Jul 21 18:07:07.652763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.652740    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mwblh"]
Jul 21 18:07:07.657725 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:07.657705    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mwblh"]
Jul 21 18:07:09.100875 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:09.100842    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d0cef02-335f-4858-b80e-310bf99d4e73" path="/var/lib/kubelet/pods/0d0cef02-335f-4858-b80e-310bf99d4e73/volumes"
Jul 21 18:07:13.491487 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:13.491457    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:13.491899 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:13.491508    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:13.513711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:13.513685    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:13.680277 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:13.680252    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-754jw"
Jul 21 18:07:34.668035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.665691    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"]
Jul 21 18:07:34.668035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.666380    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d0cef02-335f-4858-b80e-310bf99d4e73" containerName="registry-server"
Jul 21 18:07:34.668035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.666399    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d0cef02-335f-4858-b80e-310bf99d4e73" containerName="registry-server"
Jul 21 18:07:34.668035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.666544    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d0cef02-335f-4858-b80e-310bf99d4e73" containerName="registry-server"
Jul 21 18:07:34.670470 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.669838    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:07:34.673669 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.673648    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-8sfgz\""
Jul 21 18:07:34.675555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.675534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6m7k\" (UniqueName: \"kubernetes.io/projected/7137d792-3455-4afb-b827-4ff522e6ab92-kube-api-access-l6m7k\") pod \"limitador-operator-controller-manager-859d7b78bd-v5klw\" (UID: \"7137d792-3455-4afb-b827-4ff522e6ab92\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:07:34.678201 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.678174    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"]
Jul 21 18:07:34.776375 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.776350    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6m7k\" (UniqueName: \"kubernetes.io/projected/7137d792-3455-4afb-b827-4ff522e6ab92-kube-api-access-l6m7k\") pod \"limitador-operator-controller-manager-859d7b78bd-v5klw\" (UID: \"7137d792-3455-4afb-b827-4ff522e6ab92\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:07:34.790111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.790093    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6m7k\" (UniqueName: \"kubernetes.io/projected/7137d792-3455-4afb-b827-4ff522e6ab92-kube-api-access-l6m7k\") pod \"limitador-operator-controller-manager-859d7b78bd-v5klw\" (UID: \"7137d792-3455-4afb-b827-4ff522e6ab92\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:07:34.980454 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:34.980372    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:07:35.115407 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:35.115382    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"]
Jul 21 18:07:35.118251 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:07:35.118227    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7137d792_3455_4afb_b827_4ff522e6ab92.slice/crio-a3152a3c7659c3b0d276945a4d9510edc5f50e940b2215c076f9c08b808f9d89 WatchSource:0}: Error finding container a3152a3c7659c3b0d276945a4d9510edc5f50e940b2215c076f9c08b808f9d89: Status 404 returned error can't find the container with id a3152a3c7659c3b0d276945a4d9510edc5f50e940b2215c076f9c08b808f9d89
Jul 21 18:07:35.747556 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:35.747527    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw" event={"ID":"7137d792-3455-4afb-b827-4ff522e6ab92","Type":"ContainerStarted","Data":"a3152a3c7659c3b0d276945a4d9510edc5f50e940b2215c076f9c08b808f9d89"}
Jul 21 18:07:37.758318 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:37.758282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw" event={"ID":"7137d792-3455-4afb-b827-4ff522e6ab92","Type":"ContainerStarted","Data":"568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7"}
Jul 21 18:07:37.758725 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:37.758411    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:07:37.784505 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:37.784454    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw" podStartSLOduration=1.929783489 podStartE2EDuration="3.784440586s" totalImagesPullingTime="1.854657097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:07:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:07:37.783583077 +0000 UTC m=+446.042026260" watchObservedRunningTime="2026-07-21 18:07:37.784440586 +0000 UTC m=+446.042883765"
Jul 21 18:07:38.167523 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.167494    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"]
Jul 21 18:07:38.170893 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.170878    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"
Jul 21 18:07:38.173863 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.173840    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-q29hm\""
Jul 21 18:07:38.174178 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.174164    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 21 18:07:38.187352 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.187329    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"]
Jul 21 18:07:38.201772 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.201748    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqzcm\" (UniqueName: \"kubernetes.io/projected/2a949a58-5882-44ed-8071-3326ea7d49bb-kube-api-access-pqzcm\") pod \"dns-operator-controller-manager-864dd78c4c-z4qbd\" (UID: \"2a949a58-5882-44ed-8071-3326ea7d49bb\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"
Jul 21 18:07:38.302705 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.302668    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqzcm\" (UniqueName: \"kubernetes.io/projected/2a949a58-5882-44ed-8071-3326ea7d49bb-kube-api-access-pqzcm\") pod \"dns-operator-controller-manager-864dd78c4c-z4qbd\" (UID: \"2a949a58-5882-44ed-8071-3326ea7d49bb\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"
Jul 21 18:07:38.313064 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.313034    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqzcm\" (UniqueName: \"kubernetes.io/projected/2a949a58-5882-44ed-8071-3326ea7d49bb-kube-api-access-pqzcm\") pod \"dns-operator-controller-manager-864dd78c4c-z4qbd\" (UID: \"2a949a58-5882-44ed-8071-3326ea7d49bb\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"
Jul 21 18:07:38.481121 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.481039    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"
Jul 21 18:07:38.620563 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.620536    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"]
Jul 21 18:07:38.622342 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:07:38.622313    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a949a58_5882_44ed_8071_3326ea7d49bb.slice/crio-83dbb7713651fa7d8e7d1e0a69c48011ff513c3d81b78b74fb1a3628142c9c2c WatchSource:0}: Error finding container 83dbb7713651fa7d8e7d1e0a69c48011ff513c3d81b78b74fb1a3628142c9c2c: Status 404 returned error can't find the container with id 83dbb7713651fa7d8e7d1e0a69c48011ff513c3d81b78b74fb1a3628142c9c2c
Jul 21 18:07:38.763543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:38.763451    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd" event={"ID":"2a949a58-5882-44ed-8071-3326ea7d49bb","Type":"ContainerStarted","Data":"83dbb7713651fa7d8e7d1e0a69c48011ff513c3d81b78b74fb1a3628142c9c2c"}
Jul 21 18:07:41.776910 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:41.776871    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd" event={"ID":"2a949a58-5882-44ed-8071-3326ea7d49bb","Type":"ContainerStarted","Data":"2d554144a45e2dd8dbbd8c3f07ea8afc12a1581ad25744d9b91cd0457839d21a"}
Jul 21 18:07:41.777288 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:41.777023    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"
Jul 21 18:07:41.819175 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:41.819126    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd" podStartSLOduration=1.596890318 podStartE2EDuration="3.819112098s" totalImagesPullingTime="2.22222178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:07:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:07:41.816455284 +0000 UTC m=+450.074898463" watchObservedRunningTime="2026-07-21 18:07:41.819112098 +0000 UTC m=+450.077555278"
Jul 21 18:07:43.003847 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.003811    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"]
Jul 21 18:07:43.009001 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.008979    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.012365 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.012337    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-v7plk\""
Jul 21 18:07:43.020776 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.020755    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"]
Jul 21 18:07:43.043954 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.043928    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9q8m\" (UniqueName: \"kubernetes.io/projected/453379b6-7e04-44a7-bcfd-284d8e76fcd6-kube-api-access-t9q8m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.044049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.043989    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/453379b6-7e04-44a7-bcfd-284d8e76fcd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.144608 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.144564    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9q8m\" (UniqueName: \"kubernetes.io/projected/453379b6-7e04-44a7-bcfd-284d8e76fcd6-kube-api-access-t9q8m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.144744 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.144691    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/453379b6-7e04-44a7-bcfd-284d8e76fcd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.145069 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.145049    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/453379b6-7e04-44a7-bcfd-284d8e76fcd6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.154055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.154031    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9q8m\" (UniqueName: \"kubernetes.io/projected/453379b6-7e04-44a7-bcfd-284d8e76fcd6-kube-api-access-t9q8m\") pod \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.319062 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.318983    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:43.467918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.467862    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"]
Jul 21 18:07:43.470532 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:07:43.470506    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod453379b6_7e04_44a7_bcfd_284d8e76fcd6.slice/crio-6f60a23177c15cf2851e69a110343e621b3147ad852b049cd8242ccc0da97412 WatchSource:0}: Error finding container 6f60a23177c15cf2851e69a110343e621b3147ad852b049cd8242ccc0da97412: Status 404 returned error can't find the container with id 6f60a23177c15cf2851e69a110343e621b3147ad852b049cd8242ccc0da97412
Jul 21 18:07:43.786634 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:43.786584    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6" event={"ID":"453379b6-7e04-44a7-bcfd-284d8e76fcd6","Type":"ContainerStarted","Data":"6f60a23177c15cf2851e69a110343e621b3147ad852b049cd8242ccc0da97412"}
Jul 21 18:07:46.865548 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:46.865511    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"]
Jul 21 18:07:46.868933 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:46.868913    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"
Jul 21 18:07:46.875283 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:46.875230    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-7pkbr\""
Jul 21 18:07:46.882893 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:46.882869    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"]
Jul 21 18:07:46.975540 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:46.975503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-727hq\" (UniqueName: \"kubernetes.io/projected/634ddebd-0b55-4ce5-9f45-d8bc3e5c2626-kube-api-access-727hq\") pod \"authorino-operator-75cc8d5cb7-flk4v\" (UID: \"634ddebd-0b55-4ce5-9f45-d8bc3e5c2626\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"
Jul 21 18:07:47.076829 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:47.076792    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-727hq\" (UniqueName: \"kubernetes.io/projected/634ddebd-0b55-4ce5-9f45-d8bc3e5c2626-kube-api-access-727hq\") pod \"authorino-operator-75cc8d5cb7-flk4v\" (UID: \"634ddebd-0b55-4ce5-9f45-d8bc3e5c2626\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"
Jul 21 18:07:47.087407 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:47.087377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-727hq\" (UniqueName: \"kubernetes.io/projected/634ddebd-0b55-4ce5-9f45-d8bc3e5c2626-kube-api-access-727hq\") pod \"authorino-operator-75cc8d5cb7-flk4v\" (UID: \"634ddebd-0b55-4ce5-9f45-d8bc3e5c2626\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"
Jul 21 18:07:47.179552 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:47.179515    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"
Jul 21 18:07:47.339065 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:47.339028    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"]
Jul 21 18:07:47.343224 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:07:47.343189    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod634ddebd_0b55_4ce5_9f45_d8bc3e5c2626.slice/crio-1cb6b49ca0a472f025f7ae59d2d0bd83f36df88a4b3ba4464ffea61bd3da7c21 WatchSource:0}: Error finding container 1cb6b49ca0a472f025f7ae59d2d0bd83f36df88a4b3ba4464ffea61bd3da7c21: Status 404 returned error can't find the container with id 1cb6b49ca0a472f025f7ae59d2d0bd83f36df88a4b3ba4464ffea61bd3da7c21
Jul 21 18:07:47.804939 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:47.804907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v" event={"ID":"634ddebd-0b55-4ce5-9f45-d8bc3e5c2626","Type":"ContainerStarted","Data":"1cb6b49ca0a472f025f7ae59d2d0bd83f36df88a4b3ba4464ffea61bd3da7c21"}
Jul 21 18:07:48.765870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:48.765839    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:07:50.821638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:50.821586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6" event={"ID":"453379b6-7e04-44a7-bcfd-284d8e76fcd6","Type":"ContainerStarted","Data":"6726bf92a389b83ccae4c48b86eebad21dc4f3e7b37dc622d5de365992273aca"}
Jul 21 18:07:50.822055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:50.821787    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:07:50.823185 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:50.823162    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v" event={"ID":"634ddebd-0b55-4ce5-9f45-d8bc3e5c2626","Type":"ContainerStarted","Data":"ad21b8cec3e9813263362adb67d0ef8898d99bc926144c984673141e61dd3533"}
Jul 21 18:07:50.823313 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:50.823207    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"
Jul 21 18:07:50.842889 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:50.842848    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6" podStartSLOduration=2.019565551 podStartE2EDuration="8.842835064s" totalImagesPullingTime="6.823269513s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:07:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:07:50.842441835 +0000 UTC m=+459.100885015" watchObservedRunningTime="2026-07-21 18:07:50.842835064 +0000 UTC m=+459.101278243"
Jul 21 18:07:50.861937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:50.861894    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v" podStartSLOduration=1.879079473 podStartE2EDuration="4.861880828s" totalImagesPullingTime="2.982801355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:07:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:07:50.860096539 +0000 UTC m=+459.118539720" watchObservedRunningTime="2026-07-21 18:07:50.861880828 +0000 UTC m=+459.120324007"
Jul 21 18:07:52.783999 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:07:52.783962    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-z4qbd"
Jul 21 18:08:01.828868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:01.828824    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-flk4v"
Jul 21 18:08:01.829328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:01.828915    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:03.770586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.770550    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"]
Jul 21 18:08:03.770961 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.770737    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" containerName="manager" containerID="cri-o://6726bf92a389b83ccae4c48b86eebad21dc4f3e7b37dc622d5de365992273aca" gracePeriod=2
Jul 21 18:08:03.777532 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.777505    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"]
Jul 21 18:08:03.796908 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.796877    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"]
Jul 21 18:08:03.797166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.797113    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" containerName="manager" containerID="cri-o://568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7" gracePeriod=2
Jul 21 18:08:03.818039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.818009    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"]
Jul 21 18:08:03.818371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.818355    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" containerName="manager"
Jul 21 18:08:03.818371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.818372    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" containerName="manager"
Jul 21 18:08:03.818536 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.818438    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" containerName="manager"
Jul 21 18:08:03.827206 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.827182    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"]
Jul 21 18:08:03.827309 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.827290    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:03.834328 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:03.834293    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"7137d792-3455-4afb-b827-4ff522e6ab92\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-v5klw\": pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:03.839411 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:03.839381    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:03.841061 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.841026    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"]
Jul 21 18:08:03.842472 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:03.842445    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:03.846492 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:03.846461    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:03.851705 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.851684    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"]
Jul 21 18:08:03.851970 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.851958    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" containerName="manager"
Jul 21 18:08:03.852027 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.851971    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" containerName="manager"
Jul 21 18:08:03.852027 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.852019    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" containerName="manager"
Jul 21 18:08:03.854741 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.854723    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"
Jul 21 18:08:03.874018 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.873985    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"]
Jul 21 18:08:03.885623 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.885580    2578 generic.go:356] "Generic (PLEG): container finished" podID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" containerID="6726bf92a389b83ccae4c48b86eebad21dc4f3e7b37dc622d5de365992273aca" exitCode=0
Jul 21 18:08:03.900859 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:03.900827    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:03.903975 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:03.903862    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:03.920707 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.920682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh24l\" (UniqueName: \"kubernetes.io/projected/7cbc79ee-68ce-4fdf-b495-15a4abf9a08b-kube-api-access-nh24l\") pod \"limitador-operator-controller-manager-859d7b78bd-5b8ss\" (UID: \"7cbc79ee-68ce-4fdf-b495-15a4abf9a08b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"
Jul 21 18:08:03.920815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.920734    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvm7d\" (UniqueName: \"kubernetes.io/projected/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-kube-api-access-vvm7d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmbl5\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:03.920892 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:03.920871    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmbl5\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:04.022187 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.022107    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmbl5\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:04.022187 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.022153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nh24l\" (UniqueName: \"kubernetes.io/projected/7cbc79ee-68ce-4fdf-b495-15a4abf9a08b-kube-api-access-nh24l\") pod \"limitador-operator-controller-manager-859d7b78bd-5b8ss\" (UID: \"7cbc79ee-68ce-4fdf-b495-15a4abf9a08b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"
Jul 21 18:08:04.022187 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.022186    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvm7d\" (UniqueName: \"kubernetes.io/projected/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-kube-api-access-vvm7d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmbl5\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:04.022487 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.022467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmbl5\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:04.041328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.041295    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh24l\" (UniqueName: \"kubernetes.io/projected/7cbc79ee-68ce-4fdf-b495-15a4abf9a08b-kube-api-access-nh24l\") pod \"limitador-operator-controller-manager-859d7b78bd-5b8ss\" (UID: \"7cbc79ee-68ce-4fdf-b495-15a4abf9a08b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"
Jul 21 18:08:04.041328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.041321    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvm7d\" (UniqueName: \"kubernetes.io/projected/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-kube-api-access-vvm7d\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hmbl5\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:04.046414 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.046395    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:04.048479 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.048463    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:04.049780 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.049748    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:04.052386 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.052356    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:04.054902 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.054872    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:04.057367 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.057344    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:04.123292 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.123267    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9q8m\" (UniqueName: \"kubernetes.io/projected/453379b6-7e04-44a7-bcfd-284d8e76fcd6-kube-api-access-t9q8m\") pod \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") "
Jul 21 18:08:04.123502 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.123314    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6m7k\" (UniqueName: \"kubernetes.io/projected/7137d792-3455-4afb-b827-4ff522e6ab92-kube-api-access-l6m7k\") pod \"7137d792-3455-4afb-b827-4ff522e6ab92\" (UID: \"7137d792-3455-4afb-b827-4ff522e6ab92\") "
Jul 21 18:08:04.123502 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.123341    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/453379b6-7e04-44a7-bcfd-284d8e76fcd6-extensions-socket-volume\") pod \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\" (UID: \"453379b6-7e04-44a7-bcfd-284d8e76fcd6\") "
Jul 21 18:08:04.123976 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.123948    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/453379b6-7e04-44a7-bcfd-284d8e76fcd6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "453379b6-7e04-44a7-bcfd-284d8e76fcd6" (UID: "453379b6-7e04-44a7-bcfd-284d8e76fcd6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:08:04.125476 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.125447    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7137d792-3455-4afb-b827-4ff522e6ab92-kube-api-access-l6m7k" (OuterVolumeSpecName: "kube-api-access-l6m7k") pod "7137d792-3455-4afb-b827-4ff522e6ab92" (UID: "7137d792-3455-4afb-b827-4ff522e6ab92"). InnerVolumeSpecName "kube-api-access-l6m7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:08:04.125629 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.125609    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/453379b6-7e04-44a7-bcfd-284d8e76fcd6-kube-api-access-t9q8m" (OuterVolumeSpecName: "kube-api-access-t9q8m") pod "453379b6-7e04-44a7-bcfd-284d8e76fcd6" (UID: "453379b6-7e04-44a7-bcfd-284d8e76fcd6"). InnerVolumeSpecName "kube-api-access-t9q8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:08:04.205485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.205458    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:04.213512 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.213494    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"
Jul 21 18:08:04.224819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.224774    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/453379b6-7e04-44a7-bcfd-284d8e76fcd6-extensions-socket-volume\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:08:04.224819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.224800    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9q8m\" (UniqueName: \"kubernetes.io/projected/453379b6-7e04-44a7-bcfd-284d8e76fcd6-kube-api-access-t9q8m\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:08:04.224819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.224816    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6m7k\" (UniqueName: \"kubernetes.io/projected/7137d792-3455-4afb-b827-4ff522e6ab92-kube-api-access-l6m7k\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:08:04.372156 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.372128    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"]
Jul 21 18:08:04.374424 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:08:04.374397    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod516483ec_f944_4ed3_bcf0_c7d3a0262f4b.slice/crio-ea8d6934bc7ff49157b0c61bde36d15ecd4b71b35bad5c47e4177c38bfdf3d13 WatchSource:0}: Error finding container ea8d6934bc7ff49157b0c61bde36d15ecd4b71b35bad5c47e4177c38bfdf3d13: Status 404 returned error can't find the container with id ea8d6934bc7ff49157b0c61bde36d15ecd4b71b35bad5c47e4177c38bfdf3d13
Jul 21 18:08:04.375160 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.375136    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"]
Jul 21 18:08:04.376603 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:08:04.376570    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cbc79ee_68ce_4fdf_b495_15a4abf9a08b.slice/crio-68236b432ec316ffc427e58f1e642197f208fb801c28721b1a12205d08500379 WatchSource:0}: Error finding container 68236b432ec316ffc427e58f1e642197f208fb801c28721b1a12205d08500379: Status 404 returned error can't find the container with id 68236b432ec316ffc427e58f1e642197f208fb801c28721b1a12205d08500379
Jul 21 18:08:04.893552 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.893518    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:04.894027 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.893534    2578 scope.go:117] "RemoveContainer" containerID="6726bf92a389b83ccae4c48b86eebad21dc4f3e7b37dc622d5de365992273aca"
Jul 21 18:08:04.895340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.895302    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss" event={"ID":"7cbc79ee-68ce-4fdf-b495-15a4abf9a08b","Type":"ContainerStarted","Data":"086dbc431f3c7939a761e23c45bc3333605f990be191d5fa5161798c2c775f44"}
Jul 21 18:08:04.895455 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.895373    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"
Jul 21 18:08:04.895455 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.895402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss" event={"ID":"7cbc79ee-68ce-4fdf-b495-15a4abf9a08b","Type":"ContainerStarted","Data":"68236b432ec316ffc427e58f1e642197f208fb801c28721b1a12205d08500379"}
Jul 21 18:08:04.897167 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.897132    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:04.897295 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.897168    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5" event={"ID":"516483ec-f944-4ed3-bcf0-c7d3a0262f4b","Type":"ContainerStarted","Data":"991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708"}
Jul 21 18:08:04.897295 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.897220    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5" event={"ID":"516483ec-f944-4ed3-bcf0-c7d3a0262f4b","Type":"ContainerStarted","Data":"ea8d6934bc7ff49157b0c61bde36d15ecd4b71b35bad5c47e4177c38bfdf3d13"}
Jul 21 18:08:04.897402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.897315    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:04.898645 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.898624    2578 generic.go:356] "Generic (PLEG): container finished" podID="7137d792-3455-4afb-b827-4ff522e6ab92" containerID="568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7" exitCode=0
Jul 21 18:08:04.898752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.898647    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:04.899459 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.899435    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:04.902657 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.902646    2578 scope.go:117] "RemoveContainer" containerID="568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7"
Jul 21 18:08:04.910994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.910976    2578 scope.go:117] "RemoveContainer" containerID="568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7"
Jul 21 18:08:04.911219 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.911201    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7\": container with ID starting with 568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7 not found: ID does not exist" containerID="568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7"
Jul 21 18:08:04.911263 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.911226    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7"} err="failed to get container status \"568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7\": rpc error: code = NotFound desc = could not find container \"568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7\": container with ID starting with 568464fc8c34bd73d2aad479364e93417b1aa1a5339834e56d15634b7a2453d7 not found: ID does not exist"
Jul 21 18:08:04.921071 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.921035    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss" podStartSLOduration=1.921021261 podStartE2EDuration="1.921021261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:08:04.919169727 +0000 UTC m=+473.177612907" watchObservedRunningTime="2026-07-21 18:08:04.921021261 +0000 UTC m=+473.179464441"
Jul 21 18:08:04.941167 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:04.941129    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5" podStartSLOduration=1.941116906 podStartE2EDuration="1.941116906s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:08:04.940673419 +0000 UTC m=+473.199116602" watchObservedRunningTime="2026-07-21 18:08:04.941116906 +0000 UTC m=+473.199560086"
Jul 21 18:08:04.943681 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.943658    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:04.946461 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.946444    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:04.949007 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.948989    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-v5klw\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5klw"
Jul 21 18:08:04.951058 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:04.951035    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-c65x6\" is forbidden: User \"system:node:ip-10-0-140-73.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-73.ec2.internal' and this object" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-c65x6"
Jul 21 18:08:05.103009 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:05.102934    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="453379b6-7e04-44a7-bcfd-284d8e76fcd6" path="/var/lib/kubelet/pods/453379b6-7e04-44a7-bcfd-284d8e76fcd6/volumes"
Jul 21 18:08:05.103252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:05.103239    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7137d792-3455-4afb-b827-4ff522e6ab92" path="/var/lib/kubelet/pods/7137d792-3455-4afb-b827-4ff522e6ab92/volumes"
Jul 21 18:08:15.907483 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:15.907451    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-5b8ss"
Jul 21 18:08:15.907912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:15.907529    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:20.662146 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.662109    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"]
Jul 21 18:08:20.662703 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.662384    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5" podUID="516483ec-f944-4ed3-bcf0-c7d3a0262f4b" containerName="manager" containerID="cri-o://991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708" gracePeriod=10
Jul 21 18:08:20.912143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.912118    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:20.969448 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.969370    2578 generic.go:356] "Generic (PLEG): container finished" podID="516483ec-f944-4ed3-bcf0-c7d3a0262f4b" containerID="991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708" exitCode=0
Jul 21 18:08:20.969448 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.969415    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"
Jul 21 18:08:20.969651 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.969449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5" event={"ID":"516483ec-f944-4ed3-bcf0-c7d3a0262f4b","Type":"ContainerDied","Data":"991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708"}
Jul 21 18:08:20.969651 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.969503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5" event={"ID":"516483ec-f944-4ed3-bcf0-c7d3a0262f4b","Type":"ContainerDied","Data":"ea8d6934bc7ff49157b0c61bde36d15ecd4b71b35bad5c47e4177c38bfdf3d13"}
Jul 21 18:08:20.969651 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.969527    2578 scope.go:117] "RemoveContainer" containerID="991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708"
Jul 21 18:08:20.977726 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.977715    2578 scope.go:117] "RemoveContainer" containerID="991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708"
Jul 21 18:08:20.977987 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:08:20.977968    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708\": container with ID starting with 991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708 not found: ID does not exist" containerID="991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708"
Jul 21 18:08:20.978055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:20.977997    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708"} err="failed to get container status \"991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708\": rpc error: code = NotFound desc = could not find container \"991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708\": container with ID starting with 991b7871c4a2880cd9c00f065322e046e5612b8daa3660713208eff6d751e708 not found: ID does not exist"
Jul 21 18:08:21.071632 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.071580    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-extensions-socket-volume\") pod \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") "
Jul 21 18:08:21.071789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.071639    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvm7d\" (UniqueName: \"kubernetes.io/projected/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-kube-api-access-vvm7d\") pod \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\" (UID: \"516483ec-f944-4ed3-bcf0-c7d3a0262f4b\") "
Jul 21 18:08:21.072005 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.071976    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "516483ec-f944-4ed3-bcf0-c7d3a0262f4b" (UID: "516483ec-f944-4ed3-bcf0-c7d3a0262f4b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:08:21.073698 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.073672    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-kube-api-access-vvm7d" (OuterVolumeSpecName: "kube-api-access-vvm7d") pod "516483ec-f944-4ed3-bcf0-c7d3a0262f4b" (UID: "516483ec-f944-4ed3-bcf0-c7d3a0262f4b"). InnerVolumeSpecName "kube-api-access-vvm7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:08:21.172960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.172926    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-extensions-socket-volume\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:08:21.172960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.172956    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvm7d\" (UniqueName: \"kubernetes.io/projected/516483ec-f944-4ed3-bcf0-c7d3a0262f4b-kube-api-access-vvm7d\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:08:21.299778 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.299701    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"]
Jul 21 18:08:21.316452 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:21.316417    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hmbl5"]
Jul 21 18:08:23.101302 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:23.101269    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="516483ec-f944-4ed3-bcf0-c7d3a0262f4b" path="/var/lib/kubelet/pods/516483ec-f944-4ed3-bcf0-c7d3a0262f4b/volumes"
Jul 21 18:08:37.112537 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.112502    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"]
Jul 21 18:08:37.112974 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.112822    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="516483ec-f944-4ed3-bcf0-c7d3a0262f4b" containerName="manager"
Jul 21 18:08:37.112974 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.112833    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="516483ec-f944-4ed3-bcf0-c7d3a0262f4b" containerName="manager"
Jul 21 18:08:37.112974 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.112880    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="516483ec-f944-4ed3-bcf0-c7d3a0262f4b" containerName="manager"
Jul 21 18:08:37.114824 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.114808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.117584 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.117560    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-q2nwz\""
Jul 21 18:08:37.129612 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.129572    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"]
Jul 21 18:08:37.201362 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201328    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201522 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201383    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201522 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201453    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkcrp\" (UniqueName: \"kubernetes.io/projected/0d30f709-825f-4f5c-9b68-da46b32a7d5b-kube-api-access-hkcrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201522 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201719 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201719 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201568    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201719 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201586    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201719 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201619    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.201719 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.201685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.302969 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.302934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.302969 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.302969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303159 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303159 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303122    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkcrp\" (UniqueName: \"kubernetes.io/projected/0d30f709-825f-4f5c-9b68-da46b32a7d5b-kube-api-access-hkcrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303160    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303204    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303399 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303399 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303399 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303721 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303693    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303736    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.303851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.303835    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.304117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.304095    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.305365 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.305335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.305513 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.305498    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.311711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.311689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d30f709-825f-4f5c-9b68-da46b32a7d5b-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.311791 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.311718    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkcrp\" (UniqueName: \"kubernetes.io/projected/0d30f709-825f-4f5c-9b68-da46b32a7d5b-kube-api-access-hkcrp\") pod \"maas-default-gateway-openshift-default-56654dcf7-t5x67\" (UID: \"0d30f709-825f-4f5c-9b68-da46b32a7d5b\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.425324 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.425292    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:37.564245 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.564137    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"]
Jul 21 18:08:37.566749 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:08:37.566719    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d30f709_825f_4f5c_9b68_da46b32a7d5b.slice/crio-e62b9d3ca05ad7d1c8697a928a4aab9fdc128209fa3f5c0baead8af3a5e22e19 WatchSource:0}: Error finding container e62b9d3ca05ad7d1c8697a928a4aab9fdc128209fa3f5c0baead8af3a5e22e19: Status 404 returned error can't find the container with id e62b9d3ca05ad7d1c8697a928a4aab9fdc128209fa3f5c0baead8af3a5e22e19
Jul 21 18:08:37.569168 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.569129    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:08:37.569246 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.569202    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:08:37.569246 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:37.569235    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:08:38.048809 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:38.048768    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" event={"ID":"0d30f709-825f-4f5c-9b68-da46b32a7d5b","Type":"ContainerStarted","Data":"c67f0939ff27d44c3b9fecd90443dacd9358cd735a615ac7ed983b6a883e540d"}
Jul 21 18:08:38.048989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:38.048865    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" event={"ID":"0d30f709-825f-4f5c-9b68-da46b32a7d5b","Type":"ContainerStarted","Data":"e62b9d3ca05ad7d1c8697a928a4aab9fdc128209fa3f5c0baead8af3a5e22e19"}
Jul 21 18:08:38.069971 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:38.069919    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" podStartSLOduration=1.069906647 podStartE2EDuration="1.069906647s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:08:38.069621525 +0000 UTC m=+506.328064699" watchObservedRunningTime="2026-07-21 18:08:38.069906647 +0000 UTC m=+506.328349826"
Jul 21 18:08:38.426273 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:38.426238    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:38.430584 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:38.430558    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:39.053761 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:39.053729    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:39.054882 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:39.054860    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:08:56.844981 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.844945    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jnshl"]
Jul 21 18:08:56.848553 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.848537    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:56.851321 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.851305    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 21 18:08:56.851522 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.851497    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-xcwkl\""
Jul 21 18:08:56.856905 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.856885    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jnshl"]
Jul 21 18:08:56.942409 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.942376    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jnshl"]
Jul 21 18:08:56.987236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.987203    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mprz\" (UniqueName: \"kubernetes.io/projected/9774c8b5-200b-4a66-8628-59972af0f4b4-kube-api-access-6mprz\") pod \"limitador-limitador-64c94fd64f-jnshl\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:56.987389 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:56.987308    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9774c8b5-200b-4a66-8628-59972af0f4b4-config-file\") pod \"limitador-limitador-64c94fd64f-jnshl\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:57.088222 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.088185    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9774c8b5-200b-4a66-8628-59972af0f4b4-config-file\") pod \"limitador-limitador-64c94fd64f-jnshl\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:57.088400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.088242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mprz\" (UniqueName: \"kubernetes.io/projected/9774c8b5-200b-4a66-8628-59972af0f4b4-kube-api-access-6mprz\") pod \"limitador-limitador-64c94fd64f-jnshl\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:57.088848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.088827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9774c8b5-200b-4a66-8628-59972af0f4b4-config-file\") pod \"limitador-limitador-64c94fd64f-jnshl\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:57.097801 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.097739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mprz\" (UniqueName: \"kubernetes.io/projected/9774c8b5-200b-4a66-8628-59972af0f4b4-kube-api-access-6mprz\") pod \"limitador-limitador-64c94fd64f-jnshl\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:57.160012 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.159976    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:08:57.293733 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.293707    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jnshl"]
Jul 21 18:08:57.296525 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:08:57.296496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9774c8b5_200b_4a66_8628_59972af0f4b4.slice/crio-16cfe2a31a05b8047b290a9dd59746363c5caacb6e79c76ff39ced83dac8594c WatchSource:0}: Error finding container 16cfe2a31a05b8047b290a9dd59746363c5caacb6e79c76ff39ced83dac8594c: Status 404 returned error can't find the container with id 16cfe2a31a05b8047b290a9dd59746363c5caacb6e79c76ff39ced83dac8594c
Jul 21 18:08:57.675997 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.675959    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-zwbbx"]
Jul 21 18:08:57.678891 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.678868    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zwbbx"
Jul 21 18:08:57.681671 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.681651    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lcp7w\""
Jul 21 18:08:57.694034 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.694008    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zwbbx"]
Jul 21 18:08:57.795261 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.795228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w46s6\" (UniqueName: \"kubernetes.io/projected/34b1484c-f0de-4523-98f2-88cb3fa86be2-kube-api-access-w46s6\") pod \"authorino-58cd955675-zwbbx\" (UID: \"34b1484c-f0de-4523-98f2-88cb3fa86be2\") " pod="kuadrant-system/authorino-58cd955675-zwbbx"
Jul 21 18:08:57.896021 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.895976    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w46s6\" (UniqueName: \"kubernetes.io/projected/34b1484c-f0de-4523-98f2-88cb3fa86be2-kube-api-access-w46s6\") pod \"authorino-58cd955675-zwbbx\" (UID: \"34b1484c-f0de-4523-98f2-88cb3fa86be2\") " pod="kuadrant-system/authorino-58cd955675-zwbbx"
Jul 21 18:08:57.909134 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.909108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w46s6\" (UniqueName: \"kubernetes.io/projected/34b1484c-f0de-4523-98f2-88cb3fa86be2-kube-api-access-w46s6\") pod \"authorino-58cd955675-zwbbx\" (UID: \"34b1484c-f0de-4523-98f2-88cb3fa86be2\") " pod="kuadrant-system/authorino-58cd955675-zwbbx"
Jul 21 18:08:57.988512 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:57.988433    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zwbbx"
Jul 21 18:08:58.137897 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:58.137368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl" event={"ID":"9774c8b5-200b-4a66-8628-59972af0f4b4","Type":"ContainerStarted","Data":"16cfe2a31a05b8047b290a9dd59746363c5caacb6e79c76ff39ced83dac8594c"}
Jul 21 18:08:58.159079 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:58.159041    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-zwbbx"]
Jul 21 18:08:58.163165 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:08:58.163123    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34b1484c_f0de_4523_98f2_88cb3fa86be2.slice/crio-46508be4b76786151332bd1dd292e7d984a76dbef184d2265af7adb73453b807 WatchSource:0}: Error finding container 46508be4b76786151332bd1dd292e7d984a76dbef184d2265af7adb73453b807: Status 404 returned error can't find the container with id 46508be4b76786151332bd1dd292e7d984a76dbef184d2265af7adb73453b807
Jul 21 18:08:59.142226 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:08:59.142191    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zwbbx" event={"ID":"34b1484c-f0de-4523-98f2-88cb3fa86be2","Type":"ContainerStarted","Data":"46508be4b76786151332bd1dd292e7d984a76dbef184d2265af7adb73453b807"}
Jul 21 18:09:00.151074 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:00.151026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl" event={"ID":"9774c8b5-200b-4a66-8628-59972af0f4b4","Type":"ContainerStarted","Data":"5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f"}
Jul 21 18:09:00.151676 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:00.151185    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:09:00.171733 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:00.171161    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl" podStartSLOduration=1.5662216949999999 podStartE2EDuration="4.171141379s" totalImagesPullingTime="2.604919684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:00.169113784 +0000 UTC m=+528.427556966" watchObservedRunningTime="2026-07-21 18:09:00.171141379 +0000 UTC m=+528.429584560"
Jul 21 18:09:02.925196 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:02.920290    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zwbbx"]
Jul 21 18:09:03.165219 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:03.165180    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zwbbx" event={"ID":"34b1484c-f0de-4523-98f2-88cb3fa86be2","Type":"ContainerStarted","Data":"0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2"}
Jul 21 18:09:03.180163 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:03.180069    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-zwbbx" podStartSLOduration=2.167889524 podStartE2EDuration="6.180052781s" totalImagesPullingTime="4.012163257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:03.180009281 +0000 UTC m=+531.438452462" watchObservedRunningTime="2026-07-21 18:09:03.180052781 +0000 UTC m=+531.438495961"
Jul 21 18:09:04.169218 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:04.169181    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-zwbbx" podUID="34b1484c-f0de-4523-98f2-88cb3fa86be2" containerName="authorino" containerID="cri-o://0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2" gracePeriod=30
Jul 21 18:09:04.412744 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:04.412722    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zwbbx"
Jul 21 18:09:04.449690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:04.449617    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w46s6\" (UniqueName: \"kubernetes.io/projected/34b1484c-f0de-4523-98f2-88cb3fa86be2-kube-api-access-w46s6\") pod \"34b1484c-f0de-4523-98f2-88cb3fa86be2\" (UID: \"34b1484c-f0de-4523-98f2-88cb3fa86be2\") "
Jul 21 18:09:04.451649 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:04.451582    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34b1484c-f0de-4523-98f2-88cb3fa86be2-kube-api-access-w46s6" (OuterVolumeSpecName: "kube-api-access-w46s6") pod "34b1484c-f0de-4523-98f2-88cb3fa86be2" (UID: "34b1484c-f0de-4523-98f2-88cb3fa86be2"). InnerVolumeSpecName "kube-api-access-w46s6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:09:04.550347 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:04.550321    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w46s6\" (UniqueName: \"kubernetes.io/projected/34b1484c-f0de-4523-98f2-88cb3fa86be2-kube-api-access-w46s6\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:09:05.174550 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.174516    2578 generic.go:356] "Generic (PLEG): container finished" podID="34b1484c-f0de-4523-98f2-88cb3fa86be2" containerID="0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2" exitCode=0
Jul 21 18:09:05.174986 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.174557    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-zwbbx"
Jul 21 18:09:05.174986 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.174627    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zwbbx" event={"ID":"34b1484c-f0de-4523-98f2-88cb3fa86be2","Type":"ContainerDied","Data":"0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2"}
Jul 21 18:09:05.174986 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.174668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-zwbbx" event={"ID":"34b1484c-f0de-4523-98f2-88cb3fa86be2","Type":"ContainerDied","Data":"46508be4b76786151332bd1dd292e7d984a76dbef184d2265af7adb73453b807"}
Jul 21 18:09:05.174986 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.174685    2578 scope.go:117] "RemoveContainer" containerID="0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2"
Jul 21 18:09:05.182974 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.182956    2578 scope.go:117] "RemoveContainer" containerID="0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2"
Jul 21 18:09:05.183236 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:09:05.183219    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2\": container with ID starting with 0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2 not found: ID does not exist" containerID="0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2"
Jul 21 18:09:05.183288 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.183244    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2"} err="failed to get container status \"0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2\": rpc error: code = NotFound desc = could not find container \"0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2\": container with ID starting with 0fa5667f4fae50d9c9571cf868983b7dfea933fdae717c01d21ce3d1433e82b2 not found: ID does not exist"
Jul 21 18:09:05.192416 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.192384    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-zwbbx"]
Jul 21 18:09:05.194429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:05.194407    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-zwbbx"]
Jul 21 18:09:07.101400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:07.101369    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34b1484c-f0de-4523-98f2-88cb3fa86be2" path="/var/lib/kubelet/pods/34b1484c-f0de-4523-98f2-88cb3fa86be2/volumes"
Jul 21 18:09:11.157152 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:11.157125    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:09:11.912790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:11.912757    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jnshl"]
Jul 21 18:09:11.912956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:11.912933    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl" podUID="9774c8b5-200b-4a66-8628-59972af0f4b4" containerName="limitador" containerID="cri-o://5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f" gracePeriod=30
Jul 21 18:09:12.854586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:12.854560    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:09:12.917782 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:12.917751    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9774c8b5-200b-4a66-8628-59972af0f4b4-config-file\") pod \"9774c8b5-200b-4a66-8628-59972af0f4b4\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") "
Jul 21 18:09:12.917940 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:12.917810    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mprz\" (UniqueName: \"kubernetes.io/projected/9774c8b5-200b-4a66-8628-59972af0f4b4-kube-api-access-6mprz\") pod \"9774c8b5-200b-4a66-8628-59972af0f4b4\" (UID: \"9774c8b5-200b-4a66-8628-59972af0f4b4\") "
Jul 21 18:09:12.918091 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:12.918068    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9774c8b5-200b-4a66-8628-59972af0f4b4-config-file" (OuterVolumeSpecName: "config-file") pod "9774c8b5-200b-4a66-8628-59972af0f4b4" (UID: "9774c8b5-200b-4a66-8628-59972af0f4b4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:09:12.919854 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:12.919831    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9774c8b5-200b-4a66-8628-59972af0f4b4-kube-api-access-6mprz" (OuterVolumeSpecName: "kube-api-access-6mprz") pod "9774c8b5-200b-4a66-8628-59972af0f4b4" (UID: "9774c8b5-200b-4a66-8628-59972af0f4b4"). InnerVolumeSpecName "kube-api-access-6mprz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:09:13.018570 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.018487    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mprz\" (UniqueName: \"kubernetes.io/projected/9774c8b5-200b-4a66-8628-59972af0f4b4-kube-api-access-6mprz\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:09:13.018570 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.018516    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/9774c8b5-200b-4a66-8628-59972af0f4b4-config-file\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:09:13.211037 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.211002    2578 generic.go:356] "Generic (PLEG): container finished" podID="9774c8b5-200b-4a66-8628-59972af0f4b4" containerID="5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f" exitCode=0
Jul 21 18:09:13.211188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.211038    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl" event={"ID":"9774c8b5-200b-4a66-8628-59972af0f4b4","Type":"ContainerDied","Data":"5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f"}
Jul 21 18:09:13.211188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.211099    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl" event={"ID":"9774c8b5-200b-4a66-8628-59972af0f4b4","Type":"ContainerDied","Data":"16cfe2a31a05b8047b290a9dd59746363c5caacb6e79c76ff39ced83dac8594c"}
Jul 21 18:09:13.211188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.211113    2578 scope.go:117] "RemoveContainer" containerID="5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f"
Jul 21 18:09:13.211188 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.211048    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jnshl"
Jul 21 18:09:13.218879 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.218855    2578 scope.go:117] "RemoveContainer" containerID="5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f"
Jul 21 18:09:13.219129 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:09:13.219109    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f\": container with ID starting with 5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f not found: ID does not exist" containerID="5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f"
Jul 21 18:09:13.219187 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.219138    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f"} err="failed to get container status \"5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f\": rpc error: code = NotFound desc = could not find container \"5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f\": container with ID starting with 5d08196e675b25cb9e73290098c1b9fd72d6ca712fdbe13c76999bd5cba61c7f not found: ID does not exist"
Jul 21 18:09:13.230081 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.230053    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jnshl"]
Jul 21 18:09:13.234573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:13.234548    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jnshl"]
Jul 21 18:09:15.101753 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:15.101714    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9774c8b5-200b-4a66-8628-59972af0f4b4" path="/var/lib/kubelet/pods/9774c8b5-200b-4a66-8628-59972af0f4b4/volumes"
Jul 21 18:09:16.177218 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.177186    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"]
Jul 21 18:09:16.177696 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.177677    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9774c8b5-200b-4a66-8628-59972af0f4b4" containerName="limitador"
Jul 21 18:09:16.177769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.177697    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9774c8b5-200b-4a66-8628-59972af0f4b4" containerName="limitador"
Jul 21 18:09:16.177769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.177722    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34b1484c-f0de-4523-98f2-88cb3fa86be2" containerName="authorino"
Jul 21 18:09:16.177769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.177731    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="34b1484c-f0de-4523-98f2-88cb3fa86be2" containerName="authorino"
Jul 21 18:09:16.177918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.177814    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9774c8b5-200b-4a66-8628-59972af0f4b4" containerName="limitador"
Jul 21 18:09:16.177918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.177826    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="34b1484c-f0de-4523-98f2-88cb3fa86be2" containerName="authorino"
Jul 21 18:09:16.181161 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.181135    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.184081 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.184054    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 21 18:09:16.185247 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.185220    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 18:09:16.185345 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.185273    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 21 18:09:16.185403 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.185346    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-xlmpm\""
Jul 21 18:09:16.191263 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.191242    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"]
Jul 21 18:09:16.242811 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.242788    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b4e5056-e80f-407d-924c-fa0d40a5dd9a-data\") pod \"postgres-69fb4594cf-p6gtg\" (UID: \"7b4e5056-e80f-407d-924c-fa0d40a5dd9a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.242912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.242846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jl9h7\" (UniqueName: \"kubernetes.io/projected/7b4e5056-e80f-407d-924c-fa0d40a5dd9a-kube-api-access-jl9h7\") pod \"postgres-69fb4594cf-p6gtg\" (UID: \"7b4e5056-e80f-407d-924c-fa0d40a5dd9a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.344014 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.343983    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jl9h7\" (UniqueName: \"kubernetes.io/projected/7b4e5056-e80f-407d-924c-fa0d40a5dd9a-kube-api-access-jl9h7\") pod \"postgres-69fb4594cf-p6gtg\" (UID: \"7b4e5056-e80f-407d-924c-fa0d40a5dd9a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.344159 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.344034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b4e5056-e80f-407d-924c-fa0d40a5dd9a-data\") pod \"postgres-69fb4594cf-p6gtg\" (UID: \"7b4e5056-e80f-407d-924c-fa0d40a5dd9a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.344344 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.344328    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/7b4e5056-e80f-407d-924c-fa0d40a5dd9a-data\") pod \"postgres-69fb4594cf-p6gtg\" (UID: \"7b4e5056-e80f-407d-924c-fa0d40a5dd9a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.352278 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.352260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jl9h7\" (UniqueName: \"kubernetes.io/projected/7b4e5056-e80f-407d-924c-fa0d40a5dd9a-kube-api-access-jl9h7\") pod \"postgres-69fb4594cf-p6gtg\" (UID: \"7b4e5056-e80f-407d-924c-fa0d40a5dd9a\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.492270 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.492173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:16.619909 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:16.619886    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"]
Jul 21 18:09:16.622469 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:09:16.622439    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b4e5056_e80f_407d_924c_fa0d40a5dd9a.slice/crio-046dfdabefbad2d8cfed73226697e2a438cb39f01b161da329c6a82ada41fcc3 WatchSource:0}: Error finding container 046dfdabefbad2d8cfed73226697e2a438cb39f01b161da329c6a82ada41fcc3: Status 404 returned error can't find the container with id 046dfdabefbad2d8cfed73226697e2a438cb39f01b161da329c6a82ada41fcc3
Jul 21 18:09:17.230213 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:17.230180    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg" event={"ID":"7b4e5056-e80f-407d-924c-fa0d40a5dd9a","Type":"ContainerStarted","Data":"046dfdabefbad2d8cfed73226697e2a438cb39f01b161da329c6a82ada41fcc3"}
Jul 21 18:09:22.256988 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:22.256955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg" event={"ID":"7b4e5056-e80f-407d-924c-fa0d40a5dd9a","Type":"ContainerStarted","Data":"a8f4f5eae28c1de8a2768a4a4e7f07eba9a119f96f35455fa86559b3ad1870cb"}
Jul 21 18:09:22.257347 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:22.257033    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:22.276709 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:22.276660    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg" podStartSLOduration=1.25082438 podStartE2EDuration="6.276648639s" totalImagesPullingTime="5.025824259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:09:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:22.276039559 +0000 UTC m=+550.534482740" watchObservedRunningTime="2026-07-21 18:09:22.276648639 +0000 UTC m=+550.535091818"
Jul 21 18:09:28.290567 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:28.290536    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-p6gtg"
Jul 21 18:09:28.852281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:28.852249    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-46dl9"]
Jul 21 18:09:28.857468 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:28.857446    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-46dl9"
Jul 21 18:09:28.860447 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:28.860418    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-lcp7w\""
Jul 21 18:09:28.860740 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:28.860721    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-46dl9"]
Jul 21 18:09:28.957953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:28.957922    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9ckcw\" (UniqueName: \"kubernetes.io/projected/c9f9dc85-c5cd-42f1-8579-2b739e5b7406-kube-api-access-9ckcw\") pod \"authorino-5d4b67c446-46dl9\" (UID: \"c9f9dc85-c5cd-42f1-8579-2b739e5b7406\") " pod="kuadrant-system/authorino-5d4b67c446-46dl9"
Jul 21 18:09:29.058809 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.058765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9ckcw\" (UniqueName: \"kubernetes.io/projected/c9f9dc85-c5cd-42f1-8579-2b739e5b7406-kube-api-access-9ckcw\") pod \"authorino-5d4b67c446-46dl9\" (UID: \"c9f9dc85-c5cd-42f1-8579-2b739e5b7406\") " pod="kuadrant-system/authorino-5d4b67c446-46dl9"
Jul 21 18:09:29.070701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.070668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9ckcw\" (UniqueName: \"kubernetes.io/projected/c9f9dc85-c5cd-42f1-8579-2b739e5b7406-kube-api-access-9ckcw\") pod \"authorino-5d4b67c446-46dl9\" (UID: \"c9f9dc85-c5cd-42f1-8579-2b739e5b7406\") " pod="kuadrant-system/authorino-5d4b67c446-46dl9"
Jul 21 18:09:29.084948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.084915    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-46dl9"]
Jul 21 18:09:29.085163 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.085150    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-46dl9"
Jul 21 18:09:29.122957 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.122881    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54b66447c4-6mqj5"]
Jul 21 18:09:29.127378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.127352    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b66447c4-6mqj5"
Jul 21 18:09:29.138060 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.138033    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54b66447c4-6mqj5"]
Jul 21 18:09:29.218242 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.218207    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b66447c4-6mqj5"]
Jul 21 18:09:29.218483 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:09:29.218461    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-vdz7k], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-54b66447c4-6mqj5" podUID="3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7"
Jul 21 18:09:29.221377 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.221361    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-46dl9"]
Jul 21 18:09:29.224146 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:09:29.224122    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc9f9dc85_c5cd_42f1_8579_2b739e5b7406.slice/crio-83be424e1ed083e083f71fef782a5d05cf510b1243c80fcbc969cc76123c7ef4 WatchSource:0}: Error finding container 83be424e1ed083e083f71fef782a5d05cf510b1243c80fcbc969cc76123c7ef4: Status 404 returned error can't find the container with id 83be424e1ed083e083f71fef782a5d05cf510b1243c80fcbc969cc76123c7ef4
Jul 21 18:09:29.245506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.245480    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-588dd8d994-4b6z6"]
Jul 21 18:09:29.248926 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.248911    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.251778 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.251760    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 21 18:09:29.260954 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.260934    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vdz7k\" (UniqueName: \"kubernetes.io/projected/3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7-kube-api-access-vdz7k\") pod \"authorino-54b66447c4-6mqj5\" (UID: \"3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7\") " pod="kuadrant-system/authorino-54b66447c4-6mqj5"
Jul 21 18:09:29.261172 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.261152    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-588dd8d994-4b6z6"]
Jul 21 18:09:29.287223 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.287195    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-46dl9" event={"ID":"c9f9dc85-c5cd-42f1-8579-2b739e5b7406","Type":"ContainerStarted","Data":"83be424e1ed083e083f71fef782a5d05cf510b1243c80fcbc969cc76123c7ef4"}
Jul 21 18:09:29.287223 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.287221    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b66447c4-6mqj5"
Jul 21 18:09:29.293386 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.293367    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b66447c4-6mqj5"
Jul 21 18:09:29.362334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.362307    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/848baf5c-7ca3-43ff-a3ae-510d57fbb468-tls-cert\") pod \"authorino-588dd8d994-4b6z6\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") " pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.362433 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.362351    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhmhx\" (UniqueName: \"kubernetes.io/projected/848baf5c-7ca3-43ff-a3ae-510d57fbb468-kube-api-access-lhmhx\") pod \"authorino-588dd8d994-4b6z6\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") " pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.362478 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.362435    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vdz7k\" (UniqueName: \"kubernetes.io/projected/3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7-kube-api-access-vdz7k\") pod \"authorino-54b66447c4-6mqj5\" (UID: \"3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7\") " pod="kuadrant-system/authorino-54b66447c4-6mqj5"
Jul 21 18:09:29.370793 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.370768    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vdz7k\" (UniqueName: \"kubernetes.io/projected/3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7-kube-api-access-vdz7k\") pod \"authorino-54b66447c4-6mqj5\" (UID: \"3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7\") " pod="kuadrant-system/authorino-54b66447c4-6mqj5"
Jul 21 18:09:29.463635 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.463610    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vdz7k\" (UniqueName: \"kubernetes.io/projected/3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7-kube-api-access-vdz7k\") pod \"3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7\" (UID: \"3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7\") "
Jul 21 18:09:29.463757 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.463739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhmhx\" (UniqueName: \"kubernetes.io/projected/848baf5c-7ca3-43ff-a3ae-510d57fbb468-kube-api-access-lhmhx\") pod \"authorino-588dd8d994-4b6z6\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") " pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.463830 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.463815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/848baf5c-7ca3-43ff-a3ae-510d57fbb468-tls-cert\") pod \"authorino-588dd8d994-4b6z6\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") " pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.465638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.465586    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7-kube-api-access-vdz7k" (OuterVolumeSpecName: "kube-api-access-vdz7k") pod "3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7" (UID: "3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7"). InnerVolumeSpecName "kube-api-access-vdz7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:09:29.466125 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.466108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/848baf5c-7ca3-43ff-a3ae-510d57fbb468-tls-cert\") pod \"authorino-588dd8d994-4b6z6\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") " pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.472564 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.472540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhmhx\" (UniqueName: \"kubernetes.io/projected/848baf5c-7ca3-43ff-a3ae-510d57fbb468-kube-api-access-lhmhx\") pod \"authorino-588dd8d994-4b6z6\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") " pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.558060 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.558037    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:09:29.564930 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.564909    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vdz7k\" (UniqueName: \"kubernetes.io/projected/3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7-kube-api-access-vdz7k\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:09:29.690055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:29.690029    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-588dd8d994-4b6z6"]
Jul 21 18:09:29.692278 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:09:29.692248    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod848baf5c_7ca3_43ff_a3ae_510d57fbb468.slice/crio-b2ef9e4984846022304a32e03239a7ddfc45354287c9dc6e9b1bf269d8f5ed17 WatchSource:0}: Error finding container b2ef9e4984846022304a32e03239a7ddfc45354287c9dc6e9b1bf269d8f5ed17: Status 404 returned error can't find the container with id b2ef9e4984846022304a32e03239a7ddfc45354287c9dc6e9b1bf269d8f5ed17
Jul 21 18:09:30.294485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.294391    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-46dl9" event={"ID":"c9f9dc85-c5cd-42f1-8579-2b739e5b7406","Type":"ContainerStarted","Data":"d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551"}
Jul 21 18:09:30.294485 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.294419    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-46dl9" podUID="c9f9dc85-c5cd-42f1-8579-2b739e5b7406" containerName="authorino" containerID="cri-o://d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551" gracePeriod=30
Jul 21 18:09:30.296018 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.295997    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54b66447c4-6mqj5"
Jul 21 18:09:30.296123 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.296035    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588dd8d994-4b6z6" event={"ID":"848baf5c-7ca3-43ff-a3ae-510d57fbb468","Type":"ContainerStarted","Data":"bd9c389490710648773b3073515be62da96599d28fd082cb7b64a0b96ef6209e"}
Jul 21 18:09:30.296176 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.296120    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588dd8d994-4b6z6" event={"ID":"848baf5c-7ca3-43ff-a3ae-510d57fbb468","Type":"ContainerStarted","Data":"b2ef9e4984846022304a32e03239a7ddfc45354287c9dc6e9b1bf269d8f5ed17"}
Jul 21 18:09:30.311948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.311911    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-46dl9" podStartSLOduration=1.9532273789999999 podStartE2EDuration="2.311899057s" totalImagesPullingTime="358.671678ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:09:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:30.311275517 +0000 UTC m=+558.569718709" watchObservedRunningTime="2026-07-21 18:09:30.311899057 +0000 UTC m=+558.570342243"
Jul 21 18:09:30.330016 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.329978    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-588dd8d994-4b6z6" podStartSLOduration=1.039884076 podStartE2EDuration="1.329964975s" totalImagesPullingTime="290.080899ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:09:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:30.328635151 +0000 UTC m=+558.587078335" watchObservedRunningTime="2026-07-21 18:09:30.329964975 +0000 UTC m=+558.588408154"
Jul 21 18:09:30.389715 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.389685    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-54b66447c4-6mqj5"]
Jul 21 18:09:30.404026 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.404004    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-54b66447c4-6mqj5"]
Jul 21 18:09:30.545072 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.545012    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-46dl9"
Jul 21 18:09:30.675177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.675148    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9ckcw\" (UniqueName: \"kubernetes.io/projected/c9f9dc85-c5cd-42f1-8579-2b739e5b7406-kube-api-access-9ckcw\") pod \"c9f9dc85-c5cd-42f1-8579-2b739e5b7406\" (UID: \"c9f9dc85-c5cd-42f1-8579-2b739e5b7406\") "
Jul 21 18:09:30.677151 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.677127    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9f9dc85-c5cd-42f1-8579-2b739e5b7406-kube-api-access-9ckcw" (OuterVolumeSpecName: "kube-api-access-9ckcw") pod "c9f9dc85-c5cd-42f1-8579-2b739e5b7406" (UID: "c9f9dc85-c5cd-42f1-8579-2b739e5b7406"). InnerVolumeSpecName "kube-api-access-9ckcw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:09:30.776736 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:30.776702    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9ckcw\" (UniqueName: \"kubernetes.io/projected/c9f9dc85-c5cd-42f1-8579-2b739e5b7406-kube-api-access-9ckcw\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:09:31.102511 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.102476    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7" path="/var/lib/kubelet/pods/3c4c63d1-fbb5-4578-b2a4-80cbff1dbdc7/volumes"
Jul 21 18:09:31.301585 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.301553    2578 generic.go:356] "Generic (PLEG): container finished" podID="c9f9dc85-c5cd-42f1-8579-2b739e5b7406" containerID="d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551" exitCode=0
Jul 21 18:09:31.301988 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.301612    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-46dl9"
Jul 21 18:09:31.301988 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.301630    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-46dl9" event={"ID":"c9f9dc85-c5cd-42f1-8579-2b739e5b7406","Type":"ContainerDied","Data":"d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551"}
Jul 21 18:09:31.301988 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.301676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-46dl9" event={"ID":"c9f9dc85-c5cd-42f1-8579-2b739e5b7406","Type":"ContainerDied","Data":"83be424e1ed083e083f71fef782a5d05cf510b1243c80fcbc969cc76123c7ef4"}
Jul 21 18:09:31.301988 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.301694    2578 scope.go:117] "RemoveContainer" containerID="d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551"
Jul 21 18:09:31.309680 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.309662    2578 scope.go:117] "RemoveContainer" containerID="d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551"
Jul 21 18:09:31.309941 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:09:31.309923    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551\": container with ID starting with d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551 not found: ID does not exist" containerID="d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551"
Jul 21 18:09:31.309987 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.309950    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551"} err="failed to get container status \"d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551\": rpc error: code = NotFound desc = could not find container \"d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551\": container with ID starting with d29dff73f4ca35f0e3d2619c3a03be1034fa91fc4e774982954814eca2ac1551 not found: ID does not exist"
Jul 21 18:09:31.318153 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.318131    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-46dl9"]
Jul 21 18:09:31.325476 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:31.325458    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-46dl9"]
Jul 21 18:09:33.102484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:33.102452    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9f9dc85-c5cd-42f1-8579-2b739e5b7406" path="/var/lib/kubelet/pods/c9f9dc85-c5cd-42f1-8579-2b739e5b7406/volumes"
Jul 21 18:09:35.081225 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.081190    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67bfd8dfdf-vv946"]
Jul 21 18:09:35.081581 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.081542    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9f9dc85-c5cd-42f1-8579-2b739e5b7406" containerName="authorino"
Jul 21 18:09:35.081581 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.081554    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9f9dc85-c5cd-42f1-8579-2b739e5b7406" containerName="authorino"
Jul 21 18:09:35.081663 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.081638    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9f9dc85-c5cd-42f1-8579-2b739e5b7406" containerName="authorino"
Jul 21 18:09:35.313690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.313661    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bfd8dfdf-vv946"]
Jul 21 18:09:35.313866 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.313703    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.316816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.316791    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 18:09:35.316959 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.316853    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zgld2\""
Jul 21 18:09:35.316959 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.316883    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 18:09:35.319963 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.319937    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d9f8c6488-9vmtq"]
Jul 21 18:09:35.330913 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.330894    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-9vmtq"]
Jul 21 18:09:35.331022 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.330972    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.424583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.424553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c710ca2f-f52e-4f36-bd28-4ee1e9905889-webhook-cert\") pod \"maas-controller-d9f8c6488-9vmtq\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") " pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.424752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.424584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a24a2b53-8385-4447-a251-c7603bbdc733-webhook-cert\") pod \"maas-controller-67bfd8dfdf-vv946\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") " pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.424799 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.424779    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8t52s\" (UniqueName: \"kubernetes.io/projected/c710ca2f-f52e-4f36-bd28-4ee1e9905889-kube-api-access-8t52s\") pod \"maas-controller-d9f8c6488-9vmtq\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") " pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.424907 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.424886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bg2rr\" (UniqueName: \"kubernetes.io/projected/a24a2b53-8385-4447-a251-c7603bbdc733-kube-api-access-bg2rr\") pod \"maas-controller-67bfd8dfdf-vv946\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") " pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.526098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.526066    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bg2rr\" (UniqueName: \"kubernetes.io/projected/a24a2b53-8385-4447-a251-c7603bbdc733-kube-api-access-bg2rr\") pod \"maas-controller-67bfd8dfdf-vv946\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") " pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.526271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.526113    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c710ca2f-f52e-4f36-bd28-4ee1e9905889-webhook-cert\") pod \"maas-controller-d9f8c6488-9vmtq\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") " pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.526271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.526129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a24a2b53-8385-4447-a251-c7603bbdc733-webhook-cert\") pod \"maas-controller-67bfd8dfdf-vv946\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") " pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.526271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.526166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8t52s\" (UniqueName: \"kubernetes.io/projected/c710ca2f-f52e-4f36-bd28-4ee1e9905889-kube-api-access-8t52s\") pod \"maas-controller-d9f8c6488-9vmtq\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") " pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.528603 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.528576    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c710ca2f-f52e-4f36-bd28-4ee1e9905889-webhook-cert\") pod \"maas-controller-d9f8c6488-9vmtq\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") " pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.528731 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.528613    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a24a2b53-8385-4447-a251-c7603bbdc733-webhook-cert\") pod \"maas-controller-67bfd8dfdf-vv946\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") " pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.536008 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.535990    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8t52s\" (UniqueName: \"kubernetes.io/projected/c710ca2f-f52e-4f36-bd28-4ee1e9905889-kube-api-access-8t52s\") pod \"maas-controller-d9f8c6488-9vmtq\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") " pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.536174 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.536157    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bg2rr\" (UniqueName: \"kubernetes.io/projected/a24a2b53-8385-4447-a251-c7603bbdc733-kube-api-access-bg2rr\") pod \"maas-controller-67bfd8dfdf-vv946\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") " pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.625790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.625718    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:35.640638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.640617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:35.774530 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.774380    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67bfd8dfdf-vv946"]
Jul 21 18:09:35.777575 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:09:35.777541    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda24a2b53_8385_4447_a251_c7603bbdc733.slice/crio-c35c4cb9fb7f1269109b23c5a07eaa933192b99612fbc8d3169c082bbff2e10f WatchSource:0}: Error finding container c35c4cb9fb7f1269109b23c5a07eaa933192b99612fbc8d3169c082bbff2e10f: Status 404 returned error can't find the container with id c35c4cb9fb7f1269109b23c5a07eaa933192b99612fbc8d3169c082bbff2e10f
Jul 21 18:09:35.820124 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:35.820101    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-9vmtq"]
Jul 21 18:09:35.822478 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:09:35.822449    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc710ca2f_f52e_4f36_bd28_4ee1e9905889.slice/crio-76884e61da0db3a5c4930d5a97cdc35066916284a5726f307627493fd1d63f6e WatchSource:0}: Error finding container 76884e61da0db3a5c4930d5a97cdc35066916284a5726f307627493fd1d63f6e: Status 404 returned error can't find the container with id 76884e61da0db3a5c4930d5a97cdc35066916284a5726f307627493fd1d63f6e
Jul 21 18:09:36.326834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:36.326798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-9vmtq" event={"ID":"c710ca2f-f52e-4f36-bd28-4ee1e9905889","Type":"ContainerStarted","Data":"76884e61da0db3a5c4930d5a97cdc35066916284a5726f307627493fd1d63f6e"}
Jul 21 18:09:36.327810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:36.327791    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bfd8dfdf-vv946" event={"ID":"a24a2b53-8385-4447-a251-c7603bbdc733","Type":"ContainerStarted","Data":"c35c4cb9fb7f1269109b23c5a07eaa933192b99612fbc8d3169c082bbff2e10f"}
Jul 21 18:09:39.348975 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:39.348941    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-9vmtq" event={"ID":"c710ca2f-f52e-4f36-bd28-4ee1e9905889","Type":"ContainerStarted","Data":"9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611"}
Jul 21 18:09:39.349498 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:39.349052    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:39.350476 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:39.350455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bfd8dfdf-vv946" event={"ID":"a24a2b53-8385-4447-a251-c7603bbdc733","Type":"ContainerStarted","Data":"6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6"}
Jul 21 18:09:39.350623 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:39.350601    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:39.382914 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:39.382865    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d9f8c6488-9vmtq" podStartSLOduration=1.864714983 podStartE2EDuration="4.382850856s" totalImagesPullingTime="2.518135873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:39.379246724 +0000 UTC m=+567.637689904" watchObservedRunningTime="2026-07-21 18:09:39.382850856 +0000 UTC m=+567.641294036"
Jul 21 18:09:39.413806 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:39.413752    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-67bfd8dfdf-vv946" podStartSLOduration=1.840291169 podStartE2EDuration="4.413737395s" totalImagesPullingTime="2.573446226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:09:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:09:39.413063572 +0000 UTC m=+567.671506762" watchObservedRunningTime="2026-07-21 18:09:39.413737395 +0000 UTC m=+567.672180575"
Jul 21 18:09:50.357545 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.357515    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:09:50.357999 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.357580    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:50.452081 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.452053    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67bfd8dfdf-vv946"]
Jul 21 18:09:50.452267 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.452226    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-67bfd8dfdf-vv946" podUID="a24a2b53-8385-4447-a251-c7603bbdc733" containerName="manager" containerID="cri-o://6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6" gracePeriod=30
Jul 21 18:09:50.690659 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.690633    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:50.749422 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.749390    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a24a2b53-8385-4447-a251-c7603bbdc733-webhook-cert\") pod \"a24a2b53-8385-4447-a251-c7603bbdc733\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") "
Jul 21 18:09:50.749581 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.749461    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bg2rr\" (UniqueName: \"kubernetes.io/projected/a24a2b53-8385-4447-a251-c7603bbdc733-kube-api-access-bg2rr\") pod \"a24a2b53-8385-4447-a251-c7603bbdc733\" (UID: \"a24a2b53-8385-4447-a251-c7603bbdc733\") "
Jul 21 18:09:50.751544 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.751521    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a24a2b53-8385-4447-a251-c7603bbdc733-kube-api-access-bg2rr" (OuterVolumeSpecName: "kube-api-access-bg2rr") pod "a24a2b53-8385-4447-a251-c7603bbdc733" (UID: "a24a2b53-8385-4447-a251-c7603bbdc733"). InnerVolumeSpecName "kube-api-access-bg2rr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:09:50.751617 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.751525    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a24a2b53-8385-4447-a251-c7603bbdc733-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a24a2b53-8385-4447-a251-c7603bbdc733" (UID: "a24a2b53-8385-4447-a251-c7603bbdc733"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:09:50.850070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.850041    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a24a2b53-8385-4447-a251-c7603bbdc733-webhook-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:09:50.850070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:50.850066    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bg2rr\" (UniqueName: \"kubernetes.io/projected/a24a2b53-8385-4447-a251-c7603bbdc733-kube-api-access-bg2rr\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:09:51.407180 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.407150    2578 generic.go:356] "Generic (PLEG): container finished" podID="a24a2b53-8385-4447-a251-c7603bbdc733" containerID="6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6" exitCode=0
Jul 21 18:09:51.407580 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.407199    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67bfd8dfdf-vv946"
Jul 21 18:09:51.407580 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.407208    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bfd8dfdf-vv946" event={"ID":"a24a2b53-8385-4447-a251-c7603bbdc733","Type":"ContainerDied","Data":"6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6"}
Jul 21 18:09:51.407580 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.407253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67bfd8dfdf-vv946" event={"ID":"a24a2b53-8385-4447-a251-c7603bbdc733","Type":"ContainerDied","Data":"c35c4cb9fb7f1269109b23c5a07eaa933192b99612fbc8d3169c082bbff2e10f"}
Jul 21 18:09:51.407580 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.407277    2578 scope.go:117] "RemoveContainer" containerID="6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6"
Jul 21 18:09:51.415534 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.415515    2578 scope.go:117] "RemoveContainer" containerID="6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6"
Jul 21 18:09:51.415779 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:09:51.415762    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6\": container with ID starting with 6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6 not found: ID does not exist" containerID="6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6"
Jul 21 18:09:51.415837 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.415789    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6"} err="failed to get container status \"6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6\": rpc error: code = NotFound desc = could not find container \"6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6\": container with ID starting with 6db0d5e31005b7838a9f92c9b2417a901b5d9588827f12be87d339e3b6d772a6 not found: ID does not exist"
Jul 21 18:09:51.425961 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.425924    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-67bfd8dfdf-vv946"]
Jul 21 18:09:51.427821 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:51.427798    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-67bfd8dfdf-vv946"]
Jul 21 18:09:53.101539 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:09:53.101503    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a24a2b53-8385-4447-a251-c7603bbdc733" path="/var/lib/kubelet/pods/a24a2b53-8385-4447-a251-c7603bbdc733/volumes"
Jul 21 18:10:10.539350 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.539315    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"]
Jul 21 18:10:10.539790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.539672    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a24a2b53-8385-4447-a251-c7603bbdc733" containerName="manager"
Jul 21 18:10:10.539790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.539686    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a24a2b53-8385-4447-a251-c7603bbdc733" containerName="manager"
Jul 21 18:10:10.539790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.539749    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a24a2b53-8385-4447-a251-c7603bbdc733" containerName="manager"
Jul 21 18:10:10.544496 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.544478    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.547602 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.547556    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-sh2jm\""
Jul 21 18:10:10.547758 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.547578    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 21 18:10:10.555482 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.555462    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"]
Jul 21 18:10:10.616766 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.616732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ps7v\" (UniqueName: \"kubernetes.io/projected/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-kube-api-access-7ps7v\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.616937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.616791    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-openshift-service-ca\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.616937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.616853    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-maas-api-tls\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.717790 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.717757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-openshift-service-ca\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.717956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.717809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-maas-api-tls\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.717956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.717851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ps7v\" (UniqueName: \"kubernetes.io/projected/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-kube-api-access-7ps7v\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.718027 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:10:10.717991    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 21 18:10:10.718078 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:10:10.718066    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-maas-api-tls podName:9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e nodeName:}" failed. No retries permitted until 2026-07-21 18:10:11.218045664 +0000 UTC m=+599.476488834 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-maas-api-tls") pod "maas-api-75796d6899-fgmfx" (UID: "9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e") : secret "maas-api-serving-cert" not found
Jul 21 18:10:10.718384 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.718366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-openshift-service-ca\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:10.727186 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:10.727158    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ps7v\" (UniqueName: \"kubernetes.io/projected/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-kube-api-access-7ps7v\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:11.222625 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:11.222565    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-maas-api-tls\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:11.225031 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:11.225006    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e-maas-api-tls\") pod \"maas-api-75796d6899-fgmfx\" (UID: \"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e\") " pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:11.455474 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:11.455443    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:11.588503 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:11.588472    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"]
Jul 21 18:10:11.591619 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:10:11.591574    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c4c96d5_a8e2_40fe_94a4_e1a4a47b7a5e.slice/crio-84c9f44d81629b97317aadcfb26a3d493849842028bdd175ec71c9ca9bf16bc6 WatchSource:0}: Error finding container 84c9f44d81629b97317aadcfb26a3d493849842028bdd175ec71c9ca9bf16bc6: Status 404 returned error can't find the container with id 84c9f44d81629b97317aadcfb26a3d493849842028bdd175ec71c9ca9bf16bc6
Jul 21 18:10:12.506640 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:12.506582    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx" event={"ID":"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e","Type":"ContainerStarted","Data":"84c9f44d81629b97317aadcfb26a3d493849842028bdd175ec71c9ca9bf16bc6"}
Jul 21 18:10:13.419905 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:13.419879    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:10:13.422849 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:13.422829    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:10:13.518608 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:13.518562    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx" event={"ID":"9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e","Type":"ContainerStarted","Data":"ac9dc48962dd1c719245ce8979eb948903659638d451c0b916d61191432e4f23"}
Jul 21 18:10:13.518731 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:13.518714    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:13.540003 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:13.539961    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx" podStartSLOduration=1.707018843 podStartE2EDuration="3.539948088s" totalImagesPullingTime="1.832929245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:10:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:10:13.539135746 +0000 UTC m=+601.797578939" watchObservedRunningTime="2026-07-21 18:10:13.539948088 +0000 UTC m=+601.798391267"
Jul 21 18:10:19.527993 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:19.527961    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-75796d6899-fgmfx"
Jul 21 18:10:19.546959 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:19.546928    2578 generic.go:356] "Generic (PLEG): container finished" podID="0d30f709-825f-4f5c-9b68-da46b32a7d5b" containerID="c67f0939ff27d44c3b9fecd90443dacd9358cd735a615ac7ed983b6a883e540d" exitCode=0
Jul 21 18:10:19.547098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:19.547003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" event={"ID":"0d30f709-825f-4f5c-9b68-da46b32a7d5b","Type":"ContainerDied","Data":"c67f0939ff27d44c3b9fecd90443dacd9358cd735a615ac7ed983b6a883e540d"}
Jul 21 18:10:19.547347 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:19.547333    2578 scope.go:117] "RemoveContainer" containerID="c67f0939ff27d44c3b9fecd90443dacd9358cd735a615ac7ed983b6a883e540d"
Jul 21 18:10:19.548282 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:19.548254    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:10:19.548346 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:19.548321    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:10:19.548380 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:19.548351    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:10:20.554896 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:20.554856    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" event={"ID":"0d30f709-825f-4f5c-9b68-da46b32a7d5b","Type":"ContainerStarted","Data":"2de3c757bd8c40922beaa19fed69b0fe576d96ac5b6f4d7c772fd3b03e51e6a3"}
Jul 21 18:10:21.426283 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:21.426240    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:10:21.427319 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:21.427292    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t5x67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 21 18:10:21.427453 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:21.427352    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" podUID="0d30f709-825f-4f5c-9b68-da46b32a7d5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 21 18:10:22.426182 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:22.426149    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t5x67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 21 18:10:22.426569 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:22.426224    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" podUID="0d30f709-825f-4f5c-9b68-da46b32a7d5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 21 18:10:23.425955 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:23.425918    2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-t5x67 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Jul 21 18:10:23.426131 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:23.425992    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67" podUID="0d30f709-825f-4f5c-9b68-da46b32a7d5b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Jul 21 18:10:24.053954 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:24.053923    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:10:24.461097 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:24.461066    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:10:24.580948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:24.580915    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-t5x67"
Jul 21 18:10:39.029788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.029715    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"]
Jul 21 18:10:39.032038 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.032018    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.034789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.034766    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 18:10:39.034904 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.034884    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-g5lsg\""
Jul 21 18:10:39.035962 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.035945    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 18:10:39.036236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.036220    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 21 18:10:39.044834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.044813    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"]
Jul 21 18:10:39.174478 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.174442    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39224ca6-488e-4777-94c3-15c5a7bd64df-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.174658 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.174484    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.174658 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.174537    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.174658 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.174574    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfr7t\" (UniqueName: \"kubernetes.io/projected/39224ca6-488e-4777-94c3-15c5a7bd64df-kube-api-access-dfr7t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.174658 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.174634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.174794 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.174663    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275091 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275050    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39224ca6-488e-4777-94c3-15c5a7bd64df-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275091 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275091    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275111    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfr7t\" (UniqueName: \"kubernetes.io/projected/39224ca6-488e-4777-94c3-15c5a7bd64df-kube-api-access-dfr7t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275192    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275683 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275651    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275787 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275686    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.275787 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.275656    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.277479 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.277455    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/39224ca6-488e-4777-94c3-15c5a7bd64df-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.277635 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.277556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/39224ca6-488e-4777-94c3-15c5a7bd64df-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.283661 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.283586    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfr7t\" (UniqueName: \"kubernetes.io/projected/39224ca6-488e-4777-94c3-15c5a7bd64df-kube-api-access-dfr7t\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz\" (UID: \"39224ca6-488e-4777-94c3-15c5a7bd64df\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.342583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.342547    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:39.494578 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.494503    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"]
Jul 21 18:10:39.497636 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:10:39.497605    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39224ca6_488e_4777_94c3_15c5a7bd64df.slice/crio-95a8ecc6f45f0ad0d4569636da54e1a8be05b1ecbe2ade304c419f77844acabc WatchSource:0}: Error finding container 95a8ecc6f45f0ad0d4569636da54e1a8be05b1ecbe2ade304c419f77844acabc: Status 404 returned error can't find the container with id 95a8ecc6f45f0ad0d4569636da54e1a8be05b1ecbe2ade304c419f77844acabc
Jul 21 18:10:39.646726 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:39.646686    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz" event={"ID":"39224ca6-488e-4777-94c3-15c5a7bd64df","Type":"ContainerStarted","Data":"95a8ecc6f45f0ad0d4569636da54e1a8be05b1ecbe2ade304c419f77844acabc"}
Jul 21 18:10:41.661677 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:41.661638    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz" event={"ID":"39224ca6-488e-4777-94c3-15c5a7bd64df","Type":"ContainerStarted","Data":"fb8ec5198c6f8cd3a785109957082b559841fd873e93f0f34696923e5fbbaaed"}
Jul 21 18:10:41.685993 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:41.685934    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz" podStartSLOduration=1.479186799 podStartE2EDuration="2.685915044s" totalImagesPullingTime="1.206728245s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:10:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:10:41.684207862 +0000 UTC m=+629.942651042" watchObservedRunningTime="2026-07-21 18:10:41.685915044 +0000 UTC m=+629.944358226"
Jul 21 18:10:49.342676 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:49.342633    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:49.343047 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:49.342698    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:49.354946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:49.354918    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:10:49.712191 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:10:49.712160    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz"
Jul 21 18:11:08.744981 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.744949    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"]
Jul 21 18:11:08.748163 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.748145    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.751928 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.751908    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 21 18:11:08.777399 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.777370    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"]
Jul 21 18:11:08.818898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.818863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.818898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.818897    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.819092 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.818933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.819092 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.818985    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhjjv\" (UniqueName: \"kubernetes.io/projected/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-kube-api-access-vhjjv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.819092 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.819064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.819181 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.819097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920145 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920202    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920267    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhjjv\" (UniqueName: \"kubernetes.io/projected/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-kube-api-access-vhjjv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920678 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920648    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.920786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.920726    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.922325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.922302    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.922708 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.922690    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:08.931242 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:08.931219    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhjjv\" (UniqueName: \"kubernetes.io/projected/5f9e95a6-47d6-42fb-8563-1e02e2b70bb4-kube-api-access-vhjjv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl\" (UID: \"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:09.058996 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:09.058902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:09.203110 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:09.203080    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"]
Jul 21 18:11:09.207344 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:11:09.207316    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f9e95a6_47d6_42fb_8563_1e02e2b70bb4.slice/crio-a15a19f855fcdd162e31a4860d768a4b9ff129d9d396f9d57d9af7b83e6c9c3e WatchSource:0}: Error finding container a15a19f855fcdd162e31a4860d768a4b9ff129d9d396f9d57d9af7b83e6c9c3e: Status 404 returned error can't find the container with id a15a19f855fcdd162e31a4860d768a4b9ff129d9d396f9d57d9af7b83e6c9c3e
Jul 21 18:11:09.208690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:09.208671    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 18:11:09.790985 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:09.790907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl" event={"ID":"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4","Type":"ContainerStarted","Data":"89154e99512c94f6f446efd1ce999210276dc6d327ae83985c26065a8340a5c4"}
Jul 21 18:11:09.790985 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:09.790954    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl" event={"ID":"5f9e95a6-47d6-42fb-8563-1e02e2b70bb4","Type":"ContainerStarted","Data":"a15a19f855fcdd162e31a4860d768a4b9ff129d9d396f9d57d9af7b83e6c9c3e"}
Jul 21 18:11:09.838833 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:09.838782    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl" podStartSLOduration=1.6357456099999999 podStartE2EDuration="1.83876542s" totalImagesPullingTime="203.01981ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:11:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:11:09.837828862 +0000 UTC m=+658.096272039" watchObservedRunningTime="2026-07-21 18:11:09.83876542 +0000 UTC m=+658.097208589"
Jul 21 18:11:14.980425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:14.980387    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"]
Jul 21 18:11:14.984791 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:14.984770    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:14.988020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:14.988000    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 21 18:11:15.013745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.013715    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"]
Jul 21 18:11:15.074249 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.074216    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.074431 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.074291    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.074431 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.074335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.074431 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.074353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6hf2\" (UniqueName: \"kubernetes.io/projected/1e04577c-b70c-4755-80ee-1708a3169ca2-kube-api-access-d6hf2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.074431 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.074391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.074621 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.074433    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e04577c-b70c-4755-80ee-1708a3169ca2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175379 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175340    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175550 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175550 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6hf2\" (UniqueName: \"kubernetes.io/projected/1e04577c-b70c-4755-80ee-1708a3169ca2-kube-api-access-d6hf2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175550 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175550 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175448    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e04577c-b70c-4755-80ee-1708a3169ca2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175550 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175521    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175875 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175850    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175875 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.175991 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.175918    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.177853 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.177824    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e04577c-b70c-4755-80ee-1708a3169ca2-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.177972 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.177891    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e04577c-b70c-4755-80ee-1708a3169ca2-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.194248 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.194220    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6hf2\" (UniqueName: \"kubernetes.io/projected/1e04577c-b70c-4755-80ee-1708a3169ca2-kube-api-access-d6hf2\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-9lq9z\" (UID: \"1e04577c-b70c-4755-80ee-1708a3169ca2\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.294376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.294263    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:15.434243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.434059    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"]
Jul 21 18:11:15.436878 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:11:15.436856    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e04577c_b70c_4755_80ee_1708a3169ca2.slice/crio-03ed0d83c2fc0a0f548eeda70973814659657508003accab20b2e4315b811ad6 WatchSource:0}: Error finding container 03ed0d83c2fc0a0f548eeda70973814659657508003accab20b2e4315b811ad6: Status 404 returned error can't find the container with id 03ed0d83c2fc0a0f548eeda70973814659657508003accab20b2e4315b811ad6
Jul 21 18:11:15.819802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.819764    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z" event={"ID":"1e04577c-b70c-4755-80ee-1708a3169ca2","Type":"ContainerStarted","Data":"965ec53f8dad5b39526d42023926824c2520acd5338c0730bfeef7585b4537c4"}
Jul 21 18:11:15.819972 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.819851    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z" event={"ID":"1e04577c-b70c-4755-80ee-1708a3169ca2","Type":"ContainerStarted","Data":"03ed0d83c2fc0a0f548eeda70973814659657508003accab20b2e4315b811ad6"}
Jul 21 18:11:15.850402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:15.850346    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z" podStartSLOduration=1.6402493059999999 podStartE2EDuration="1.850329661s" totalImagesPullingTime="210.080355ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:11:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:11:15.847884665 +0000 UTC m=+664.106327855" watchObservedRunningTime="2026-07-21 18:11:15.850329661 +0000 UTC m=+664.108772841"
Jul 21 18:11:19.059321 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:19.059282    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:19.059742 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:19.059347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:19.077167 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:19.077139    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:19.849819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:19.849791    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl"
Jul 21 18:11:25.295074 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:25.295030    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:25.295476 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:25.295098    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:25.307625 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:25.307583    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:11:25.876043 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:11:25.876009    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-9lq9z"
Jul 21 18:12:31.973393 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:31.973345    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-588dd8d994-4b6z6"]
Jul 21 18:12:31.973994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:31.973613    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-588dd8d994-4b6z6" podUID="848baf5c-7ca3-43ff-a3ae-510d57fbb468" containerName="authorino" containerID="cri-o://bd9c389490710648773b3073515be62da96599d28fd082cb7b64a0b96ef6209e" gracePeriod=30
Jul 21 18:12:32.170180 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.170150    2578 generic.go:356] "Generic (PLEG): container finished" podID="848baf5c-7ca3-43ff-a3ae-510d57fbb468" containerID="bd9c389490710648773b3073515be62da96599d28fd082cb7b64a0b96ef6209e" exitCode=0
Jul 21 18:12:32.170344 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.170205    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588dd8d994-4b6z6" event={"ID":"848baf5c-7ca3-43ff-a3ae-510d57fbb468","Type":"ContainerDied","Data":"bd9c389490710648773b3073515be62da96599d28fd082cb7b64a0b96ef6209e"}
Jul 21 18:12:32.228519 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.228456    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:12:32.259059 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.259031    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/848baf5c-7ca3-43ff-a3ae-510d57fbb468-tls-cert\") pod \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") "
Jul 21 18:12:32.259211 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.259145    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhmhx\" (UniqueName: \"kubernetes.io/projected/848baf5c-7ca3-43ff-a3ae-510d57fbb468-kube-api-access-lhmhx\") pod \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\" (UID: \"848baf5c-7ca3-43ff-a3ae-510d57fbb468\") "
Jul 21 18:12:32.261082 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.261045    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/848baf5c-7ca3-43ff-a3ae-510d57fbb468-kube-api-access-lhmhx" (OuterVolumeSpecName: "kube-api-access-lhmhx") pod "848baf5c-7ca3-43ff-a3ae-510d57fbb468" (UID: "848baf5c-7ca3-43ff-a3ae-510d57fbb468"). InnerVolumeSpecName "kube-api-access-lhmhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:12:32.268854 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.268824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/848baf5c-7ca3-43ff-a3ae-510d57fbb468-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "848baf5c-7ca3-43ff-a3ae-510d57fbb468" (UID: "848baf5c-7ca3-43ff-a3ae-510d57fbb468"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:12:32.360166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.360137    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhmhx\" (UniqueName: \"kubernetes.io/projected/848baf5c-7ca3-43ff-a3ae-510d57fbb468-kube-api-access-lhmhx\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:12:32.360166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:32.360164    2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/848baf5c-7ca3-43ff-a3ae-510d57fbb468-tls-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:12:33.175689 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:33.175656    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-588dd8d994-4b6z6" event={"ID":"848baf5c-7ca3-43ff-a3ae-510d57fbb468","Type":"ContainerDied","Data":"b2ef9e4984846022304a32e03239a7ddfc45354287c9dc6e9b1bf269d8f5ed17"}
Jul 21 18:12:33.175689 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:33.175681    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-588dd8d994-4b6z6"
Jul 21 18:12:33.176149 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:33.175709    2578 scope.go:117] "RemoveContainer" containerID="bd9c389490710648773b3073515be62da96599d28fd082cb7b64a0b96ef6209e"
Jul 21 18:12:33.198009 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:33.197985    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-588dd8d994-4b6z6"]
Jul 21 18:12:33.201919 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:33.201898    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-588dd8d994-4b6z6"]
Jul 21 18:12:35.102455 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:12:35.102422    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="848baf5c-7ca3-43ff-a3ae-510d57fbb468" path="/var/lib/kubelet/pods/848baf5c-7ca3-43ff-a3ae-510d57fbb468/volumes"
Jul 21 18:13:18.025704 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.025669    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-9vmtq"]
Jul 21 18:13:18.026142 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.025915    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d9f8c6488-9vmtq" podUID="c710ca2f-f52e-4f36-bd28-4ee1e9905889" containerName="manager" containerID="cri-o://9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611" gracePeriod=30
Jul 21 18:13:18.276964 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.276909    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:13:18.360114 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.360068    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8t52s\" (UniqueName: \"kubernetes.io/projected/c710ca2f-f52e-4f36-bd28-4ee1e9905889-kube-api-access-8t52s\") pod \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") "
Jul 21 18:13:18.360286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.360129    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c710ca2f-f52e-4f36-bd28-4ee1e9905889-webhook-cert\") pod \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\" (UID: \"c710ca2f-f52e-4f36-bd28-4ee1e9905889\") "
Jul 21 18:13:18.362027 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.361998    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c710ca2f-f52e-4f36-bd28-4ee1e9905889-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "c710ca2f-f52e-4f36-bd28-4ee1e9905889" (UID: "c710ca2f-f52e-4f36-bd28-4ee1e9905889"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:13:18.362027 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.362012    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c710ca2f-f52e-4f36-bd28-4ee1e9905889-kube-api-access-8t52s" (OuterVolumeSpecName: "kube-api-access-8t52s") pod "c710ca2f-f52e-4f36-bd28-4ee1e9905889" (UID: "c710ca2f-f52e-4f36-bd28-4ee1e9905889"). InnerVolumeSpecName "kube-api-access-8t52s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:13:18.378532 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.378502    2578 generic.go:356] "Generic (PLEG): container finished" podID="c710ca2f-f52e-4f36-bd28-4ee1e9905889" containerID="9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611" exitCode=0
Jul 21 18:13:18.378648 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.378552    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-9vmtq"
Jul 21 18:13:18.378648 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.378601    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-9vmtq" event={"ID":"c710ca2f-f52e-4f36-bd28-4ee1e9905889","Type":"ContainerDied","Data":"9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611"}
Jul 21 18:13:18.378730 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.378657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-9vmtq" event={"ID":"c710ca2f-f52e-4f36-bd28-4ee1e9905889","Type":"ContainerDied","Data":"76884e61da0db3a5c4930d5a97cdc35066916284a5726f307627493fd1d63f6e"}
Jul 21 18:13:18.378730 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.378680    2578 scope.go:117] "RemoveContainer" containerID="9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611"
Jul 21 18:13:18.387132 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.387113    2578 scope.go:117] "RemoveContainer" containerID="9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611"
Jul 21 18:13:18.387412 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:13:18.387392    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611\": container with ID starting with 9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611 not found: ID does not exist" containerID="9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611"
Jul 21 18:13:18.387459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.387421    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611"} err="failed to get container status \"9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611\": rpc error: code = NotFound desc = could not find container \"9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611\": container with ID starting with 9c4db2a06a4a42c8e38f6e8e6cb3defefbed85dd7d4a43b87432a6d18c1f9611 not found: ID does not exist"
Jul 21 18:13:18.401705 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.401678    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-9vmtq"]
Jul 21 18:13:18.413078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.413058    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-9vmtq"]
Jul 21 18:13:18.461339 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.461317    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8t52s\" (UniqueName: \"kubernetes.io/projected/c710ca2f-f52e-4f36-bd28-4ee1e9905889-kube-api-access-8t52s\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:13:18.461339 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:18.461338    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c710ca2f-f52e-4f36-bd28-4ee1e9905889-webhook-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:13:19.101599 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:13:19.101556    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c710ca2f-f52e-4f36-bd28-4ee1e9905889" path="/var/lib/kubelet/pods/c710ca2f-f52e-4f36-bd28-4ee1e9905889/volumes"
Jul 21 18:14:19.237932 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.237888    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d9f8c6488-cp8vp"]
Jul 21 18:14:19.238490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.238426    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c710ca2f-f52e-4f36-bd28-4ee1e9905889" containerName="manager"
Jul 21 18:14:19.238490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.238450    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c710ca2f-f52e-4f36-bd28-4ee1e9905889" containerName="manager"
Jul 21 18:14:19.238490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.238469    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="848baf5c-7ca3-43ff-a3ae-510d57fbb468" containerName="authorino"
Jul 21 18:14:19.238490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.238478    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="848baf5c-7ca3-43ff-a3ae-510d57fbb468" containerName="authorino"
Jul 21 18:14:19.238743 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.238585    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="848baf5c-7ca3-43ff-a3ae-510d57fbb468" containerName="authorino"
Jul 21 18:14:19.238743 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.238618    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c710ca2f-f52e-4f36-bd28-4ee1e9905889" containerName="manager"
Jul 21 18:14:19.241709 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.241686    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.248608 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.245043    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 18:14:19.251543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.249047    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 18:14:19.251543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.249322    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zgld2\""
Jul 21 18:14:19.252634 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.252610    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-cp8vp"]
Jul 21 18:14:19.270812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.270786    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-webhook-cert\") pod \"maas-controller-d9f8c6488-cp8vp\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") " pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.270918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.270832    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72b8z\" (UniqueName: \"kubernetes.io/projected/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-kube-api-access-72b8z\") pod \"maas-controller-d9f8c6488-cp8vp\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") " pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.371925 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.371899    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-webhook-cert\") pod \"maas-controller-d9f8c6488-cp8vp\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") " pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.372046 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.371945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72b8z\" (UniqueName: \"kubernetes.io/projected/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-kube-api-access-72b8z\") pod \"maas-controller-d9f8c6488-cp8vp\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") " pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.374273 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.374246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-webhook-cert\") pod \"maas-controller-d9f8c6488-cp8vp\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") " pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.382206 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.382182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72b8z\" (UniqueName: \"kubernetes.io/projected/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-kube-api-access-72b8z\") pod \"maas-controller-d9f8c6488-cp8vp\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") " pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.559566 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.559486    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:19.706154 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:19.706127    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-cp8vp"]
Jul 21 18:14:19.707715 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:14:19.707683    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod056d6f90_93bc_4dcb_adf2_1dbe6c5d5e31.slice/crio-f2ddb91b62219753686ce55a41958cde858c91d093ae21f79f0192944ed8e33c WatchSource:0}: Error finding container f2ddb91b62219753686ce55a41958cde858c91d093ae21f79f0192944ed8e33c: Status 404 returned error can't find the container with id f2ddb91b62219753686ce55a41958cde858c91d093ae21f79f0192944ed8e33c
Jul 21 18:14:20.654480 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:20.654443    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-cp8vp" event={"ID":"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31","Type":"ContainerStarted","Data":"6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492"}
Jul 21 18:14:20.654905 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:20.654514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:20.654905 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:20.654534    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-cp8vp" event={"ID":"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31","Type":"ContainerStarted","Data":"f2ddb91b62219753686ce55a41958cde858c91d093ae21f79f0192944ed8e33c"}
Jul 21 18:14:20.673351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:20.673302    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d9f8c6488-cp8vp" podStartSLOduration=1.079492357 podStartE2EDuration="1.673287037s" totalImagesPullingTime="593.79468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:14:20.672695426 +0000 UTC m=+848.931138607" watchObservedRunningTime="2026-07-21 18:14:20.673287037 +0000 UTC m=+848.931730220"
Jul 21 18:14:31.661603 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:31.661557    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:36.303752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.303714    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-cp8vp"]
Jul 21 18:14:36.304168 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.303930    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-d9f8c6488-cp8vp" podUID="056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" containerName="manager" containerID="cri-o://6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492" gracePeriod=30
Jul 21 18:14:36.553049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.553027    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:36.625884 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.625810    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-webhook-cert\") pod \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") "
Jul 21 18:14:36.625884 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.625849    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72b8z\" (UniqueName: \"kubernetes.io/projected/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-kube-api-access-72b8z\") pod \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\" (UID: \"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31\") "
Jul 21 18:14:36.627805 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.627778    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" (UID: "056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:14:36.627927 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.627894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-kube-api-access-72b8z" (OuterVolumeSpecName: "kube-api-access-72b8z") pod "056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" (UID: "056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31"). InnerVolumeSpecName "kube-api-access-72b8z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:14:36.726476 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.726450    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-webhook-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:14:36.726476 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.726476    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72b8z\" (UniqueName: \"kubernetes.io/projected/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31-kube-api-access-72b8z\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:14:36.726808 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.726786    2578 generic.go:356] "Generic (PLEG): container finished" podID="056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" containerID="6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492" exitCode=0
Jul 21 18:14:36.726868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.726832    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-cp8vp"
Jul 21 18:14:36.726910 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.726879    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-cp8vp" event={"ID":"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31","Type":"ContainerDied","Data":"6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492"}
Jul 21 18:14:36.726945 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.726933    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-cp8vp" event={"ID":"056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31","Type":"ContainerDied","Data":"f2ddb91b62219753686ce55a41958cde858c91d093ae21f79f0192944ed8e33c"}
Jul 21 18:14:36.726986 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.726957    2578 scope.go:117] "RemoveContainer" containerID="6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492"
Jul 21 18:14:36.735630 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.735574    2578 scope.go:117] "RemoveContainer" containerID="6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492"
Jul 21 18:14:36.735877 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:14:36.735850    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492\": container with ID starting with 6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492 not found: ID does not exist" containerID="6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492"
Jul 21 18:14:36.735934 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.735885    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492"} err="failed to get container status \"6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492\": rpc error: code = NotFound desc = could not find container \"6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492\": container with ID starting with 6febd9831d74260a33e86ad43038fc486ec2500457668a650e7fc6186d69d492 not found: ID does not exist"
Jul 21 18:14:36.748572 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.748550    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-cp8vp"]
Jul 21 18:14:36.750208 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:36.750190    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-cp8vp"]
Jul 21 18:14:37.101831 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.101796    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" path="/var/lib/kubelet/pods/056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31/volumes"
Jul 21 18:14:37.577930 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.577895    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-d9f8c6488-fq4vk"]
Jul 21 18:14:37.578297 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.578266    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" containerName="manager"
Jul 21 18:14:37.578297 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.578286    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" containerName="manager"
Jul 21 18:14:37.578372 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.578347    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="056d6f90-93bc-4dcb-adf2-1dbe6c5d5e31" containerName="manager"
Jul 21 18:14:37.581103 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.581086    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:37.585428 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.585409    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 21 18:14:37.585543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.585410    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 21 18:14:37.585543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.585457    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zgld2\""
Jul 21 18:14:37.591203 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.591180    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-fq4vk"]
Jul 21 18:14:37.634033 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.634005    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l78jr\" (UniqueName: \"kubernetes.io/projected/52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf-kube-api-access-l78jr\") pod \"maas-controller-d9f8c6488-fq4vk\" (UID: \"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf\") " pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:37.634159 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.634037    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf-webhook-cert\") pod \"maas-controller-d9f8c6488-fq4vk\" (UID: \"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf\") " pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:37.734842 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.734814    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l78jr\" (UniqueName: \"kubernetes.io/projected/52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf-kube-api-access-l78jr\") pod \"maas-controller-d9f8c6488-fq4vk\" (UID: \"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf\") " pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:37.734973 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.734849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf-webhook-cert\") pod \"maas-controller-d9f8c6488-fq4vk\" (UID: \"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf\") " pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:37.737266 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.737233    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf-webhook-cert\") pod \"maas-controller-d9f8c6488-fq4vk\" (UID: \"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf\") " pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:37.744720 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.744700    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l78jr\" (UniqueName: \"kubernetes.io/projected/52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf-kube-api-access-l78jr\") pod \"maas-controller-d9f8c6488-fq4vk\" (UID: \"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf\") " pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:37.891948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:37.891922    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:38.023801 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:38.023779    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-d9f8c6488-fq4vk"]
Jul 21 18:14:38.025538 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:14:38.025496    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52ad5a9f_8500_4452_aecd_4aaa2ccf9bdf.slice/crio-f5d498b305aae495751232d925c6e2db9e8c8f32cab1add66bea54ee5ee349d8 WatchSource:0}: Error finding container f5d498b305aae495751232d925c6e2db9e8c8f32cab1add66bea54ee5ee349d8: Status 404 returned error can't find the container with id f5d498b305aae495751232d925c6e2db9e8c8f32cab1add66bea54ee5ee349d8
Jul 21 18:14:38.739018 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:38.738985    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-fq4vk" event={"ID":"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf","Type":"ContainerStarted","Data":"f5d498b305aae495751232d925c6e2db9e8c8f32cab1add66bea54ee5ee349d8"}
Jul 21 18:14:39.745442 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:39.745408    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-d9f8c6488-fq4vk" event={"ID":"52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf","Type":"ContainerStarted","Data":"6ab3dc9681952db6a6324d1fbd113f67d78cc9661d3d0a141f239754c9fea572"}
Jul 21 18:14:39.745843 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:39.745480    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:14:39.766092 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:39.766047    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-d9f8c6488-fq4vk" podStartSLOduration=1.951397346 podStartE2EDuration="2.766033107s" totalImagesPullingTime="814.635761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:14:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:14:39.763406319 +0000 UTC m=+868.021849499" watchObservedRunningTime="2026-07-21 18:14:39.766033107 +0000 UTC m=+868.024476287"
Jul 21 18:14:50.752241 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:14:50.752214    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-d9f8c6488-fq4vk"
Jul 21 18:15:13.457960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:15:13.457927    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:15:13.458459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:15:13.458425    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:20:13.489876 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:20:13.489839    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:20:13.491841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:20:13.491820    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:23:17.432600 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.432554    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"]
Jul 21 18:23:17.435970 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.435941    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.438829 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.438809    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e0fd3e19-kserve-self-signed-certs\""
Jul 21 18:23:17.457134 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.457110    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"]
Jul 21 18:23:17.598687 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.598649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-model-cache\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.598855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.598700    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-home\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.598855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.598739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-dshm\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.598855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.598789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-tmp-dir\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.598855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.598841    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c08a72e5-0047-4514-af68-67f31ef0a2fa-tls-certs\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.599033 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.598875    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt59j\" (UniqueName: \"kubernetes.io/projected/c08a72e5-0047-4514-af68-67f31ef0a2fa-kube-api-access-qt59j\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.699819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.699732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c08a72e5-0047-4514-af68-67f31ef0a2fa-tls-certs\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.699819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.699768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt59j\" (UniqueName: \"kubernetes.io/projected/c08a72e5-0047-4514-af68-67f31ef0a2fa-kube-api-access-qt59j\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.700020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.699829    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-model-cache\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.700020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.699853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-home\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.700020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.699878    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-dshm\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.700020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.699989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-tmp-dir\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.700254 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.700235    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-home\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.700338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.700277    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-model-cache\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.700419 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.700398    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-tmp-dir\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.702133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.702105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-dshm\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.702312 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.702296    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c08a72e5-0047-4514-af68-67f31ef0a2fa-tls-certs\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.707950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.707930    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt59j\" (UniqueName: \"kubernetes.io/projected/c08a72e5-0047-4514-af68-67f31ef0a2fa-kube-api-access-qt59j\") pod \"e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") " pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.744996 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.744969    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:17.883927 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:17.883893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"]
Jul 21 18:23:17.887320 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:23:17.887292    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc08a72e5_0047_4514_af68_67f31ef0a2fa.slice/crio-49fc8aa315abb9c511df237dfe1c5bc9312d88c42fbf6716a0e9a042f32c1dda WatchSource:0}: Error finding container 49fc8aa315abb9c511df237dfe1c5bc9312d88c42fbf6716a0e9a042f32c1dda: Status 404 returned error can't find the container with id 49fc8aa315abb9c511df237dfe1c5bc9312d88c42fbf6716a0e9a042f32c1dda
Jul 21 18:23:18.095424 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:18.095343    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c" event={"ID":"c08a72e5-0047-4514-af68-67f31ef0a2fa","Type":"ContainerStarted","Data":"38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce"}
Jul 21 18:23:18.095424 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:18.095395    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c" event={"ID":"c08a72e5-0047-4514-af68-67f31ef0a2fa","Type":"ContainerStarted","Data":"49fc8aa315abb9c511df237dfe1c5bc9312d88c42fbf6716a0e9a042f32c1dda"}
Jul 21 18:23:18.113907 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:18.113856    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c" podStartSLOduration=1.113840988 podStartE2EDuration="1.113840988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:23:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:23:18.11238793 +0000 UTC m=+1386.370831109" watchObservedRunningTime="2026-07-21 18:23:18.113840988 +0000 UTC m=+1386.372284207"
Jul 21 18:23:27.745365 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:27.745279    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:27.745365 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:27.745337    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:27.757638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:27.757614    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:28.151419 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:28.151382    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:23:30.931262 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:30.931229    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"]
Jul 21 18:23:30.934906 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:30.934885    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:30.938577 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:30.938552    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-e0fd3e19-kserve-self-signed-certs\""
Jul 21 18:23:30.947254 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:30.947228    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"]
Jul 21 18:23:31.020348 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.020314    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-tmp-dir\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.020517 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.020354    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688db421-c46f-4940-9a81-9c2eb39a4065-tls-certs\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.020517 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.020460    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-dshm\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.020517 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.020490    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-model-cache\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.020665 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.020527    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzg4f\" (UniqueName: \"kubernetes.io/projected/688db421-c46f-4940-9a81-9c2eb39a4065-kube-api-access-nzg4f\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.020665 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.020557    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-home\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.121801 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.121772    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-home\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.121982 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.121820    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-tmp-dir\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.121982 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.121838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688db421-c46f-4940-9a81-9c2eb39a4065-tls-certs\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.121982 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.121877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-dshm\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.121982 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.121894    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-model-cache\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.121982 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.121920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzg4f\" (UniqueName: \"kubernetes.io/projected/688db421-c46f-4940-9a81-9c2eb39a4065-kube-api-access-nzg4f\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.122269 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.122231    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-tmp-dir\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.122361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.122338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-model-cache\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.122500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.122465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-home\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.124165 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.124145    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-dshm\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.124733 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.124712    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688db421-c46f-4940-9a81-9c2eb39a4065-tls-certs\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.130729 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.130708    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzg4f\" (UniqueName: \"kubernetes.io/projected/688db421-c46f-4940-9a81-9c2eb39a4065-kube-api-access-nzg4f\") pod \"e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") " pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.244167 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.244096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:31.378547 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:31.378497    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"]
Jul 21 18:23:31.381031 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:23:31.381002    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod688db421_c46f_4940_9a81_9c2eb39a4065.slice/crio-af6ea494d69c6a74328091c0cf9a1dc81ea72eec36eb51e37258f7ba783701ad WatchSource:0}: Error finding container af6ea494d69c6a74328091c0cf9a1dc81ea72eec36eb51e37258f7ba783701ad: Status 404 returned error can't find the container with id af6ea494d69c6a74328091c0cf9a1dc81ea72eec36eb51e37258f7ba783701ad
Jul 21 18:23:32.160859 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:32.160825    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr" event={"ID":"688db421-c46f-4940-9a81-9c2eb39a4065","Type":"ContainerStarted","Data":"3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4"}
Jul 21 18:23:32.161199 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:32.160869    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr" event={"ID":"688db421-c46f-4940-9a81-9c2eb39a4065","Type":"ContainerStarted","Data":"af6ea494d69c6a74328091c0cf9a1dc81ea72eec36eb51e37258f7ba783701ad"}
Jul 21 18:23:32.179912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:32.179855    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr" podStartSLOduration=2.179834764 podStartE2EDuration="2.179834764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:23:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:23:32.179386199 +0000 UTC m=+1400.437829380" watchObservedRunningTime="2026-07-21 18:23:32.179834764 +0000 UTC m=+1400.438277945"
Jul 21 18:23:41.244449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:41.244420    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:41.244817 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:41.244470    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:41.256871 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:41.256844    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:42.220607 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:42.220561    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:23:48.989294 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:48.989258    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"]
Jul 21 18:23:50.302146 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:23:50.302114    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"]
Jul 21 18:24:04.009628 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.009524    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c" podUID="c08a72e5-0047-4514-af68-67f31ef0a2fa" containerName="main" containerID="cri-o://38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce" gracePeriod=45
Jul 21 18:24:04.262914 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.262850    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:24:04.308803 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.308762    2578 generic.go:356] "Generic (PLEG): container finished" podID="c08a72e5-0047-4514-af68-67f31ef0a2fa" containerID="38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce" exitCode=0
Jul 21 18:24:04.308803 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.308800    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"
Jul 21 18:24:04.308999 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.308841    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c" event={"ID":"c08a72e5-0047-4514-af68-67f31ef0a2fa","Type":"ContainerDied","Data":"38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce"}
Jul 21 18:24:04.308999 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.308879    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c" event={"ID":"c08a72e5-0047-4514-af68-67f31ef0a2fa","Type":"ContainerDied","Data":"49fc8aa315abb9c511df237dfe1c5bc9312d88c42fbf6716a0e9a042f32c1dda"}
Jul 21 18:24:04.308999 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.308899    2578 scope.go:117] "RemoveContainer" containerID="38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce"
Jul 21 18:24:04.317177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.317159    2578 scope.go:117] "RemoveContainer" containerID="38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce"
Jul 21 18:24:04.317442 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:24:04.317426    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce\": container with ID starting with 38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce not found: ID does not exist" containerID="38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce"
Jul 21 18:24:04.317500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.317451    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce"} err="failed to get container status \"38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce\": rpc error: code = NotFound desc = could not find container \"38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce\": container with ID starting with 38a0bf436f0a0205f0feba738ab2a526618b9dc4dfaf11eb421bdcfa1ba56dce not found: ID does not exist"
Jul 21 18:24:04.426209 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426178    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qt59j\" (UniqueName: \"kubernetes.io/projected/c08a72e5-0047-4514-af68-67f31ef0a2fa-kube-api-access-qt59j\") pod \"c08a72e5-0047-4514-af68-67f31ef0a2fa\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") "
Jul 21 18:24:04.426400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426223    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-tmp-dir\") pod \"c08a72e5-0047-4514-af68-67f31ef0a2fa\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") "
Jul 21 18:24:04.426400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426278    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-dshm\") pod \"c08a72e5-0047-4514-af68-67f31ef0a2fa\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") "
Jul 21 18:24:04.426400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426310    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-model-cache\") pod \"c08a72e5-0047-4514-af68-67f31ef0a2fa\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") "
Jul 21 18:24:04.426400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426348    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-home\") pod \"c08a72e5-0047-4514-af68-67f31ef0a2fa\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") "
Jul 21 18:24:04.426400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426378    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c08a72e5-0047-4514-af68-67f31ef0a2fa-tls-certs\") pod \"c08a72e5-0047-4514-af68-67f31ef0a2fa\" (UID: \"c08a72e5-0047-4514-af68-67f31ef0a2fa\") "
Jul 21 18:24:04.426693 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426513    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c08a72e5-0047-4514-af68-67f31ef0a2fa" (UID: "c08a72e5-0047-4514-af68-67f31ef0a2fa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:04.426693 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426608    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-model-cache" (OuterVolumeSpecName: "model-cache") pod "c08a72e5-0047-4514-af68-67f31ef0a2fa" (UID: "c08a72e5-0047-4514-af68-67f31ef0a2fa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:04.426693 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.426625    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-home" (OuterVolumeSpecName: "home") pod "c08a72e5-0047-4514-af68-67f31ef0a2fa" (UID: "c08a72e5-0047-4514-af68-67f31ef0a2fa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:04.428505 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.428472    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-dshm" (OuterVolumeSpecName: "dshm") pod "c08a72e5-0047-4514-af68-67f31ef0a2fa" (UID: "c08a72e5-0047-4514-af68-67f31ef0a2fa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:04.428652 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.428515    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c08a72e5-0047-4514-af68-67f31ef0a2fa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c08a72e5-0047-4514-af68-67f31ef0a2fa" (UID: "c08a72e5-0047-4514-af68-67f31ef0a2fa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:24:04.428652 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.428576    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c08a72e5-0047-4514-af68-67f31ef0a2fa-kube-api-access-qt59j" (OuterVolumeSpecName: "kube-api-access-qt59j") pod "c08a72e5-0047-4514-af68-67f31ef0a2fa" (UID: "c08a72e5-0047-4514-af68-67f31ef0a2fa"). InnerVolumeSpecName "kube-api-access-qt59j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:24:04.527325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.527241    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-dshm\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:04.527325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.527275    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-model-cache\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:04.527325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.527293    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-home\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:04.527325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.527303    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c08a72e5-0047-4514-af68-67f31ef0a2fa-tls-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:04.527325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.527312    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qt59j\" (UniqueName: \"kubernetes.io/projected/c08a72e5-0047-4514-af68-67f31ef0a2fa-kube-api-access-qt59j\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:04.527325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.527321    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c08a72e5-0047-4514-af68-67f31ef0a2fa-tmp-dir\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:04.630873 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.630845    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"]
Jul 21 18:24:04.637479 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:04.637452    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e0fd3e19-kserve-5f7579df4c-2rs8c"]
Jul 21 18:24:05.102063 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.102029    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c08a72e5-0047-4514-af68-67f31ef0a2fa" path="/var/lib/kubelet/pods/c08a72e5-0047-4514-af68-67f31ef0a2fa/volumes"
Jul 21 18:24:05.321869 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.321828    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr" podUID="688db421-c46f-4940-9a81-9c2eb39a4065" containerName="main" containerID="cri-o://3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4" gracePeriod=45
Jul 21 18:24:05.578841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.578817    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:24:05.737891 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.737805    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-tmp-dir\") pod \"688db421-c46f-4940-9a81-9c2eb39a4065\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") "
Jul 21 18:24:05.737891 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.737864    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-home\") pod \"688db421-c46f-4940-9a81-9c2eb39a4065\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") "
Jul 21 18:24:05.738129 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.737912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-dshm\") pod \"688db421-c46f-4940-9a81-9c2eb39a4065\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") "
Jul 21 18:24:05.738129 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.737928    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-model-cache\") pod \"688db421-c46f-4940-9a81-9c2eb39a4065\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") "
Jul 21 18:24:05.738129 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.737960    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzg4f\" (UniqueName: \"kubernetes.io/projected/688db421-c46f-4940-9a81-9c2eb39a4065-kube-api-access-nzg4f\") pod \"688db421-c46f-4940-9a81-9c2eb39a4065\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") "
Jul 21 18:24:05.738129 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.738006    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688db421-c46f-4940-9a81-9c2eb39a4065-tls-certs\") pod \"688db421-c46f-4940-9a81-9c2eb39a4065\" (UID: \"688db421-c46f-4940-9a81-9c2eb39a4065\") "
Jul 21 18:24:05.738129 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.738065    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "688db421-c46f-4940-9a81-9c2eb39a4065" (UID: "688db421-c46f-4940-9a81-9c2eb39a4065"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:05.738382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.738183    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-home" (OuterVolumeSpecName: "home") pod "688db421-c46f-4940-9a81-9c2eb39a4065" (UID: "688db421-c46f-4940-9a81-9c2eb39a4065"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:05.738382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.738225    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-model-cache" (OuterVolumeSpecName: "model-cache") pod "688db421-c46f-4940-9a81-9c2eb39a4065" (UID: "688db421-c46f-4940-9a81-9c2eb39a4065"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:05.738382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.738337    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-tmp-dir\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:05.738382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.738357    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-home\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:05.738382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.738370    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-model-cache\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:05.740163 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.740136    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-dshm" (OuterVolumeSpecName: "dshm") pod "688db421-c46f-4940-9a81-9c2eb39a4065" (UID: "688db421-c46f-4940-9a81-9c2eb39a4065"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:24:05.740163 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.740146    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/688db421-c46f-4940-9a81-9c2eb39a4065-kube-api-access-nzg4f" (OuterVolumeSpecName: "kube-api-access-nzg4f") pod "688db421-c46f-4940-9a81-9c2eb39a4065" (UID: "688db421-c46f-4940-9a81-9c2eb39a4065"). InnerVolumeSpecName "kube-api-access-nzg4f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:24:05.740327 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.740140    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/688db421-c46f-4940-9a81-9c2eb39a4065-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "688db421-c46f-4940-9a81-9c2eb39a4065" (UID: "688db421-c46f-4940-9a81-9c2eb39a4065"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:24:05.839747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.839716    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/688db421-c46f-4940-9a81-9c2eb39a4065-dshm\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:05.839747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.839743    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzg4f\" (UniqueName: \"kubernetes.io/projected/688db421-c46f-4940-9a81-9c2eb39a4065-kube-api-access-nzg4f\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:05.839747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:05.839755    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/688db421-c46f-4940-9a81-9c2eb39a4065-tls-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:24:06.319662 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.319630    2578 generic.go:356] "Generic (PLEG): container finished" podID="688db421-c46f-4940-9a81-9c2eb39a4065" containerID="3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4" exitCode=0
Jul 21 18:24:06.320177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.319673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr" event={"ID":"688db421-c46f-4940-9a81-9c2eb39a4065","Type":"ContainerDied","Data":"3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4"}
Jul 21 18:24:06.320177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.319689    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"
Jul 21 18:24:06.320177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.319704    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr" event={"ID":"688db421-c46f-4940-9a81-9c2eb39a4065","Type":"ContainerDied","Data":"af6ea494d69c6a74328091c0cf9a1dc81ea72eec36eb51e37258f7ba783701ad"}
Jul 21 18:24:06.320177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.319727    2578 scope.go:117] "RemoveContainer" containerID="3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4"
Jul 21 18:24:06.328509 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.328344    2578 scope.go:117] "RemoveContainer" containerID="3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4"
Jul 21 18:24:06.328682 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:24:06.328663    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4\": container with ID starting with 3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4 not found: ID does not exist" containerID="3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4"
Jul 21 18:24:06.328733 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.328690    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4"} err="failed to get container status \"3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4\": rpc error: code = NotFound desc = could not find container \"3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4\": container with ID starting with 3b73f624de09672db06290c2aea4761866dc4081ddaba8854b71b289f0c0f6f4 not found: ID does not exist"
Jul 21 18:24:06.344195 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.344170    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"]
Jul 21 18:24:06.350818 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:06.350794    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-e0fd3e19-kserve-6cbbb9d487-bqlbr"]
Jul 21 18:24:07.102183 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:24:07.102149    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="688db421-c46f-4940-9a81-9c2eb39a4065" path="/var/lib/kubelet/pods/688db421-c46f-4940-9a81-9c2eb39a4065/volumes"
Jul 21 18:25:13.519765 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:25:13.519738    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:25:13.520874 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:25:13.520855    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:27:34.989301 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.989270    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"]
Jul 21 18:27:34.989898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.989636    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c08a72e5-0047-4514-af68-67f31ef0a2fa" containerName="main"
Jul 21 18:27:34.989898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.989649    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c08a72e5-0047-4514-af68-67f31ef0a2fa" containerName="main"
Jul 21 18:27:34.989898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.989670    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="688db421-c46f-4940-9a81-9c2eb39a4065" containerName="main"
Jul 21 18:27:34.989898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.989676    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="688db421-c46f-4940-9a81-9c2eb39a4065" containerName="main"
Jul 21 18:27:34.989898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.989728    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c08a72e5-0047-4514-af68-67f31ef0a2fa" containerName="main"
Jul 21 18:27:34.989898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.989739    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="688db421-c46f-4940-9a81-9c2eb39a4065" containerName="main"
Jul 21 18:27:34.992818 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.992799    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:34.995678 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:34.995658    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-24f6ad1b-openshift-default-dockercfg-c8mbr\""
Jul 21 18:27:35.019778 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.019752    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"]
Jul 21 18:27:35.106548 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106514    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-envoy\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106548 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106551    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-podinfo\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106761 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106584    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjxf5\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-kube-api-access-hjxf5\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106761 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106666    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-certs\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106761 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-data\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106761 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-credential-socket\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106892 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106768    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-socket\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106892 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106793    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-token\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.106892 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.106811    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istiod-ca-cert\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.207811 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.207779    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-envoy\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.207811 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.207812    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-podinfo\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208015 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.207856    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjxf5\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-kube-api-access-hjxf5\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208098    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-certs\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-data\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208426    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-certs\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-credential-socket\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208574    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-socket\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208930 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208628    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-token\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208930 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208656    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istiod-ca-cert\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208930 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208687    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-data\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.208930 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.208796    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-credential-socket\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.209194 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.209171    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-socket\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.209232 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.209183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istiod-ca-cert\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.210275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.210255    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-envoy\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.210655 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.210640    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-podinfo\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.219733 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.219711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-token\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.219963 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.219943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjxf5\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-kube-api-access-hjxf5\") pod \"e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") " pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.303866 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.303793    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:35.451360 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.451334    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"]
Jul 21 18:27:35.452963 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:27:35.452934    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15d771c0_a1f7_441d_bbe0_8e3b5432396f.slice/crio-18f52303e1e3b672206e322f7410ef69906d774fb4a348336b7b6df095d13f8a WatchSource:0}: Error finding container 18f52303e1e3b672206e322f7410ef69906d774fb4a348336b7b6df095d13f8a: Status 404 returned error can't find the container with id 18f52303e1e3b672206e322f7410ef69906d774fb4a348336b7b6df095d13f8a
Jul 21 18:27:35.455290 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.455240    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:27:35.455408 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.455336    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:27:35.455408 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:35.455382    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:27:36.260883 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:36.260843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" event={"ID":"15d771c0-a1f7-441d-bbe0-8e3b5432396f","Type":"ContainerStarted","Data":"0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee"}
Jul 21 18:27:36.261283 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:36.260916    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" event={"ID":"15d771c0-a1f7-441d-bbe0-8e3b5432396f","Type":"ContainerStarted","Data":"18f52303e1e3b672206e322f7410ef69906d774fb4a348336b7b6df095d13f8a"}
Jul 21 18:27:36.282997 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:36.282948    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" podStartSLOduration=2.282932548 podStartE2EDuration="2.282932548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:27:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:27:36.282911669 +0000 UTC m=+1644.541354848" watchObservedRunningTime="2026-07-21 18:27:36.282932548 +0000 UTC m=+1644.541375727"
Jul 21 18:27:36.305155 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:36.305047    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:36.306388 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:36.306367    2578 patch_prober.go:28] interesting pod/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 21 18:27:36.306467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:36.306421    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 21 18:27:37.308748 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.308716    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:37.730764 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.730730    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"]
Jul 21 18:27:37.734538 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.734519    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.737311 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.737289    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-24f6ad1b-serving-cert\""
Jul 21 18:27:37.744721 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.744696    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"]
Jul 21 18:27:37.833576 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.833546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj6c5\" (UniqueName: \"kubernetes.io/projected/04f8b089-0992-4694-bbad-95460ee3d813-kube-api-access-cj6c5\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.833775 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.833610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04f8b089-0992-4694-bbad-95460ee3d813-maas-api-tls\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.833775 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.833711    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04f8b089-0992-4694-bbad-95460ee3d813-openshift-service-ca\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.934585 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.934536    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04f8b089-0992-4694-bbad-95460ee3d813-maas-api-tls\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.934816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.934644    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04f8b089-0992-4694-bbad-95460ee3d813-openshift-service-ca\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.934816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.934720    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj6c5\" (UniqueName: \"kubernetes.io/projected/04f8b089-0992-4694-bbad-95460ee3d813-kube-api-access-cj6c5\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.935390 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.935366    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04f8b089-0992-4694-bbad-95460ee3d813-openshift-service-ca\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.937001 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.936977    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04f8b089-0992-4694-bbad-95460ee3d813-maas-api-tls\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:37.953888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:37.953860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj6c5\" (UniqueName: \"kubernetes.io/projected/04f8b089-0992-4694-bbad-95460ee3d813-kube-api-access-cj6c5\") pod \"maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:38.047525 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:38.047447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:38.184760 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:38.184659    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"]
Jul 21 18:27:38.187608 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:27:38.187565    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04f8b089_0992_4694_bbad_95460ee3d813.slice/crio-4c3b128ebcb5d2e9ae0b73b2fe4db5d3faadf84ea8a1444c200badb94aff1a96 WatchSource:0}: Error finding container 4c3b128ebcb5d2e9ae0b73b2fe4db5d3faadf84ea8a1444c200badb94aff1a96: Status 404 returned error can't find the container with id 4c3b128ebcb5d2e9ae0b73b2fe4db5d3faadf84ea8a1444c200badb94aff1a96
Jul 21 18:27:38.189218 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:38.189196    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 18:27:38.278144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:38.278113    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj" event={"ID":"04f8b089-0992-4694-bbad-95460ee3d813","Type":"ContainerStarted","Data":"4c3b128ebcb5d2e9ae0b73b2fe4db5d3faadf84ea8a1444c200badb94aff1a96"}
Jul 21 18:27:38.278447 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:38.278431    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:38.279375 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:38.279353    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:39.284506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:39.284464    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj" event={"ID":"04f8b089-0992-4694-bbad-95460ee3d813","Type":"ContainerStarted","Data":"0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331"}
Jul 21 18:27:39.303289 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:39.303244    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj" podStartSLOduration=1.842281793 podStartE2EDuration="2.303228782s" totalImagesPullingTime="460.946989ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:27:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:27:39.301903537 +0000 UTC m=+1647.560346717" watchObservedRunningTime="2026-07-21 18:27:39.303228782 +0000 UTC m=+1647.561671961"
Jul 21 18:27:40.288926 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:40.288889    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:46.298924 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:46.298895    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:27:52.350821 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:52.350730    2578 generic.go:356] "Generic (PLEG): container finished" podID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerID="0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee" exitCode=0
Jul 21 18:27:52.351195 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:52.350812    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" event={"ID":"15d771c0-a1f7-441d-bbe0-8e3b5432396f","Type":"ContainerDied","Data":"0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee"}
Jul 21 18:27:52.351251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:52.351237    2578 scope.go:117] "RemoveContainer" containerID="0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee"
Jul 21 18:27:52.352274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:52.352247    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:27:52.352326 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:52.352316    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:27:52.352360 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:52.352348    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:27:53.279729 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:53.279690    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:53.357384 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:53.357348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" event={"ID":"15d771c0-a1f7-441d-bbe0-8e3b5432396f","Type":"ContainerStarted","Data":"a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5"}
Jul 21 18:27:54.304212 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:54.304172    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:54.305449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:54.305419    2578 patch_prober.go:28] interesting pod/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 21 18:27:54.305575 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:54.305474    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 21 18:27:55.304160 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:55.304124    2578 patch_prober.go:28] interesting pod/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 21 18:27:55.304539 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:55.304175    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 21 18:27:56.304445 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:56.304411    2578 patch_prober.go:28] interesting pod/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 21 18:27:56.304831 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:56.304464    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 21 18:27:57.308834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:57.308803    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:57.309189 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:57.309104    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:27:57.309865 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:27:57.309849    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:28:13.626877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.626844    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"]
Jul 21 18:28:13.627412 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.627068    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj" podUID="04f8b089-0992-4694-bbad-95460ee3d813" containerName="maas-api" containerID="cri-o://0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331" gracePeriod=30
Jul 21 18:28:13.878281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.878209    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:28:13.969821 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.969796    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04f8b089-0992-4694-bbad-95460ee3d813-maas-api-tls\") pod \"04f8b089-0992-4694-bbad-95460ee3d813\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") "
Jul 21 18:28:13.969957 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.969827    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04f8b089-0992-4694-bbad-95460ee3d813-openshift-service-ca\") pod \"04f8b089-0992-4694-bbad-95460ee3d813\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") "
Jul 21 18:28:13.969957 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.969912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cj6c5\" (UniqueName: \"kubernetes.io/projected/04f8b089-0992-4694-bbad-95460ee3d813-kube-api-access-cj6c5\") pod \"04f8b089-0992-4694-bbad-95460ee3d813\" (UID: \"04f8b089-0992-4694-bbad-95460ee3d813\") "
Jul 21 18:28:13.970271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.970232    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04f8b089-0992-4694-bbad-95460ee3d813-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04f8b089-0992-4694-bbad-95460ee3d813" (UID: "04f8b089-0992-4694-bbad-95460ee3d813"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:28:13.972104 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.972083    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04f8b089-0992-4694-bbad-95460ee3d813-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04f8b089-0992-4694-bbad-95460ee3d813" (UID: "04f8b089-0992-4694-bbad-95460ee3d813"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:28:13.972183 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:13.972109    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04f8b089-0992-4694-bbad-95460ee3d813-kube-api-access-cj6c5" (OuterVolumeSpecName: "kube-api-access-cj6c5") pod "04f8b089-0992-4694-bbad-95460ee3d813" (UID: "04f8b089-0992-4694-bbad-95460ee3d813"). InnerVolumeSpecName "kube-api-access-cj6c5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:28:14.070614 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.070561    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cj6c5\" (UniqueName: \"kubernetes.io/projected/04f8b089-0992-4694-bbad-95460ee3d813-kube-api-access-cj6c5\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:14.070614 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.070587    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04f8b089-0992-4694-bbad-95460ee3d813-maas-api-tls\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:14.070614 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.070618    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04f8b089-0992-4694-bbad-95460ee3d813-openshift-service-ca\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:14.456491 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.456452    2578 generic.go:356] "Generic (PLEG): container finished" podID="04f8b089-0992-4694-bbad-95460ee3d813" containerID="0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331" exitCode=0
Jul 21 18:28:14.456686 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.456502    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"
Jul 21 18:28:14.456686 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.456536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj" event={"ID":"04f8b089-0992-4694-bbad-95460ee3d813","Type":"ContainerDied","Data":"0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331"}
Jul 21 18:28:14.456686 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.456608    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj" event={"ID":"04f8b089-0992-4694-bbad-95460ee3d813","Type":"ContainerDied","Data":"4c3b128ebcb5d2e9ae0b73b2fe4db5d3faadf84ea8a1444c200badb94aff1a96"}
Jul 21 18:28:14.456686 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.456631    2578 scope.go:117] "RemoveContainer" containerID="0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331"
Jul 21 18:28:14.465881 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.465865    2578 scope.go:117] "RemoveContainer" containerID="0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331"
Jul 21 18:28:14.466187 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:28:14.466167    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331\": container with ID starting with 0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331 not found: ID does not exist" containerID="0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331"
Jul 21 18:28:14.466272 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.466194    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331"} err="failed to get container status \"0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331\": rpc error: code = NotFound desc = could not find container \"0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331\": container with ID starting with 0db56cf9a39b2a5bd8ed088f64c180f408938a1021c9974e73d35fdd76818331 not found: ID does not exist"
Jul 21 18:28:14.478781 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.478755    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"]
Jul 21 18:28:14.484646 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:14.484626    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-24f6ad1b-6fff7899d4-tjjgj"]
Jul 21 18:28:15.102181 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:15.102144    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04f8b089-0992-4694-bbad-95460ee3d813" path="/var/lib/kubelet/pods/04f8b089-0992-4694-bbad-95460ee3d813/volumes"
Jul 21 18:28:18.413053 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:18.413019    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"]
Jul 21 18:28:18.413497 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:18.413294    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy" containerID="cri-o://a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5" gracePeriod=30
Jul 21 18:28:19.457396 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.457364    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"]
Jul 21 18:28:19.457868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.457777    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04f8b089-0992-4694-bbad-95460ee3d813" containerName="maas-api"
Jul 21 18:28:19.457868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.457790    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="04f8b089-0992-4694-bbad-95460ee3d813" containerName="maas-api"
Jul 21 18:28:19.457868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.457853    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="04f8b089-0992-4694-bbad-95460ee3d813" containerName="maas-api"
Jul 21 18:28:19.460448 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.460429    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.463782 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.463759    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-ee15a55f-gw-openshift-default-dockercfg-4bt54\""
Jul 21 18:28:19.476144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.476120    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"]
Jul 21 18:28:19.621897 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.621857    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-credential-socket\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.621915    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-envoy\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.621945    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4p9ct\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-kube-api-access-4p9ct\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.622000    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-certs\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.622062    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-istio-token\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.622095    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-socket\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.622123    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0507fee8-e007-4365-a255-3e513dddef9f-istiod-ca-cert\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.622148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-data\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.622217 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.622170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0507fee8-e007-4365-a255-3e513dddef9f-istio-podinfo\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-certs\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723231    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-istio-token\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-socket\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723294    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0507fee8-e007-4365-a255-3e513dddef9f-istiod-ca-cert\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-data\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0507fee8-e007-4365-a255-3e513dddef9f-istio-podinfo\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723403    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-credential-socket\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723436    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-envoy\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723484 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723460    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4p9ct\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-kube-api-access-4p9ct\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723922 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723616    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-certs\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.723922 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723632    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-socket\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.724044 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723924    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-credential-socket\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.724044 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.723988    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-data\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.724163 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.724139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0507fee8-e007-4365-a255-3e513dddef9f-istiod-ca-cert\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.725833 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.725802    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-envoy\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.725944 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.725892    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0507fee8-e007-4365-a255-3e513dddef9f-istio-podinfo\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.731662 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.731633    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-istio-token\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.731953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.731935    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4p9ct\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-kube-api-access-4p9ct\") pod \"e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") " pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.770840 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.770815    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:19.912524 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.912499    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"]
Jul 21 18:28:19.914822 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:28:19.914791    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0507fee8_e007_4365_a255_3e513dddef9f.slice/crio-7d2074492f4e5167656aed68f5f82e1029a90810b7349efff39b55a03cdee29b WatchSource:0}: Error finding container 7d2074492f4e5167656aed68f5f82e1029a90810b7349efff39b55a03cdee29b: Status 404 returned error can't find the container with id 7d2074492f4e5167656aed68f5f82e1029a90810b7349efff39b55a03cdee29b
Jul 21 18:28:19.917509 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.917472    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:28:19.917587 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.917555    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:28:19.917650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:19.917626    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:28:20.490170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:20.490131    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" event={"ID":"0507fee8-e007-4365-a255-3e513dddef9f","Type":"ContainerStarted","Data":"ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0"}
Jul 21 18:28:20.490576 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:20.490190    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" event={"ID":"0507fee8-e007-4365-a255-3e513dddef9f","Type":"ContainerStarted","Data":"7d2074492f4e5167656aed68f5f82e1029a90810b7349efff39b55a03cdee29b"}
Jul 21 18:28:20.514604 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:20.514556    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" podStartSLOduration=1.514541414 podStartE2EDuration="1.514541414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:28:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:28:20.512420289 +0000 UTC m=+1688.770863469" watchObservedRunningTime="2026-07-21 18:28:20.514541414 +0000 UTC m=+1688.772984594"
Jul 21 18:28:20.771298 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:20.771218    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:20.776103 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:20.776074    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:21.494610 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:21.494561    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:21.495466 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:21.495444    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:22.227314 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.227278    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"]
Jul 21 18:28:22.230024 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.229998    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.232917 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.232889    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-ee15a55f-serving-cert\""
Jul 21 18:28:22.240318 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.240294    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"]
Jul 21 18:28:22.347018 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.346987    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ef9960-a71b-453e-96b6-fd9438fb1331-openshift-service-ca\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.347214 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.347047    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ef9960-a71b-453e-96b6-fd9438fb1331-maas-api-tls\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.347214 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.347157    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c9kv\" (UniqueName: \"kubernetes.io/projected/d6ef9960-a71b-453e-96b6-fd9438fb1331-kube-api-access-7c9kv\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.447664 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.447627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ef9960-a71b-453e-96b6-fd9438fb1331-openshift-service-ca\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.447855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.447682    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ef9960-a71b-453e-96b6-fd9438fb1331-maas-api-tls\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.447855 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.447723    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c9kv\" (UniqueName: \"kubernetes.io/projected/d6ef9960-a71b-453e-96b6-fd9438fb1331-kube-api-access-7c9kv\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.448338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.448313    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ef9960-a71b-453e-96b6-fd9438fb1331-openshift-service-ca\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.450109 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.450090    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ef9960-a71b-453e-96b6-fd9438fb1331-maas-api-tls\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.455901 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.455874    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c9kv\" (UniqueName: \"kubernetes.io/projected/d6ef9960-a71b-453e-96b6-fd9438fb1331-kube-api-access-7c9kv\") pod \"maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.541792 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.541717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:22.680883 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:22.680788    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"]
Jul 21 18:28:22.683198 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:28:22.683172    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6ef9960_a71b_453e_96b6_fd9438fb1331.slice/crio-01568cb211dcbf1e5b54ccfa4984a2902da58108c4a2ecc2019a68e623ff5190 WatchSource:0}: Error finding container 01568cb211dcbf1e5b54ccfa4984a2902da58108c4a2ecc2019a68e623ff5190: Status 404 returned error can't find the container with id 01568cb211dcbf1e5b54ccfa4984a2902da58108c4a2ecc2019a68e623ff5190
Jul 21 18:28:23.505186 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.505144    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv" event={"ID":"d6ef9960-a71b-453e-96b6-fd9438fb1331","Type":"ContainerStarted","Data":"b8458925139094011db49c967c143cfa16bb7b99f665195aa0778b6cde71b175"}
Jul 21 18:28:23.505374 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.505246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv" event={"ID":"d6ef9960-a71b-453e-96b6-fd9438fb1331","Type":"ContainerStarted","Data":"01568cb211dcbf1e5b54ccfa4984a2902da58108c4a2ecc2019a68e623ff5190"}
Jul 21 18:28:23.505468 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.505440    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:23.522638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.522568    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv" podStartSLOduration=1.130880091 podStartE2EDuration="1.522554698s" totalImagesPullingTime="391.674607ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:28:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:28:23.522469644 +0000 UTC m=+1691.780912825" watchObservedRunningTime="2026-07-21 18:28:23.522554698 +0000 UTC m=+1691.780997878"
Jul 21 18:28:23.674387 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.674323    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:28:23.761271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761245    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjxf5\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-kube-api-access-hjxf5\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761295    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-envoy\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761331    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-certs\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761370    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-socket\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-podinfo\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761644 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761468    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-data\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761644 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761510    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istiod-ca-cert\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761770 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761673    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:28:23.761770 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761716    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-token\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761782    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-credential-socket\") pod \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\" (UID: \"15d771c0-a1f7-441d-bbe0-8e3b5432396f\") "
Jul 21 18:28:23.761877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761711    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:28:23.761993 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.761929    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:28:23.762081 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.762059    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:28:23.762183 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.762166    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.762237 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.762188    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.762237 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.762203    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.762237 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.762218    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.762574 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.762546    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-data" (OuterVolumeSpecName: "istio-data") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:28:23.763576 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.763517    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:28:23.763950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.763923    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-token" (OuterVolumeSpecName: "istio-token") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:28:23.763950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.763940    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:28:23.764079 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.763977    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-kube-api-access-hjxf5" (OuterVolumeSpecName: "kube-api-access-hjxf5") pod "15d771c0-a1f7-441d-bbe0-8e3b5432396f" (UID: "15d771c0-a1f7-441d-bbe0-8e3b5432396f"). InnerVolumeSpecName "kube-api-access-hjxf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:28:23.863070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.863040    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjxf5\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-kube-api-access-hjxf5\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.863070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.863064    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.863070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.863074    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.863294 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.863083    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:23.863294 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:23.863091    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/15d771c0-a1f7-441d-bbe0-8e3b5432396f-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:24.511902 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.511864    2578 generic.go:356] "Generic (PLEG): container finished" podID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerID="a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5" exitCode=0
Jul 21 18:28:24.512078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.511921    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"
Jul 21 18:28:24.512078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.511957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" event={"ID":"15d771c0-a1f7-441d-bbe0-8e3b5432396f","Type":"ContainerDied","Data":"a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5"}
Jul 21 18:28:24.512078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.512003    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26" event={"ID":"15d771c0-a1f7-441d-bbe0-8e3b5432396f","Type":"ContainerDied","Data":"18f52303e1e3b672206e322f7410ef69906d774fb4a348336b7b6df095d13f8a"}
Jul 21 18:28:24.512078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.512026    2578 scope.go:117] "RemoveContainer" containerID="a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5"
Jul 21 18:28:24.521239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.521222    2578 scope.go:117] "RemoveContainer" containerID="0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee"
Jul 21 18:28:24.529319 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.529301    2578 scope.go:117] "RemoveContainer" containerID="a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5"
Jul 21 18:28:24.529557 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:28:24.529539    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5\": container with ID starting with a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5 not found: ID does not exist" containerID="a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5"
Jul 21 18:28:24.529625 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.529565    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5"} err="failed to get container status \"a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5\": rpc error: code = NotFound desc = could not find container \"a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5\": container with ID starting with a7340ad32807264c9370d23bbb2b74bd47da052e829ebc2ee5c63f497bf2aea5 not found: ID does not exist"
Jul 21 18:28:24.529625 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.529580    2578 scope.go:117] "RemoveContainer" containerID="0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee"
Jul 21 18:28:24.529825 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:28:24.529808    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee\": container with ID starting with 0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee not found: ID does not exist" containerID="0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee"
Jul 21 18:28:24.529873 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.529830    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee"} err="failed to get container status \"0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee\": rpc error: code = NotFound desc = could not find container \"0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee\": container with ID starting with 0cd37ab78cb2e94d596dd5865d90b217b2375a190cc33fd3e073ea26ec7d60ee not found: ID does not exist"
Jul 21 18:28:24.534886 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.534865    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"]
Jul 21 18:28:24.538084 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:24.538066    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-24f6ad1b-openshift-default-677ffd8d9f-lkw26"]
Jul 21 18:28:25.102611 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:25.102559    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" path="/var/lib/kubelet/pods/15d771c0-a1f7-441d-bbe0-8e3b5432396f/volumes"
Jul 21 18:28:29.517772 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:29.517733    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:36.496041 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.495957    2578 patch_prober.go:28] interesting pod/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 21 18:28:36.496041 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.496027    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 21 18:28:36.570565 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.570476    2578 generic.go:356] "Generic (PLEG): container finished" podID="0507fee8-e007-4365-a255-3e513dddef9f" containerID="ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0" exitCode=0
Jul 21 18:28:36.570565 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.570541    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" event={"ID":"0507fee8-e007-4365-a255-3e513dddef9f","Type":"ContainerDied","Data":"ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0"}
Jul 21 18:28:36.571032 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.571012    2578 scope.go:117] "RemoveContainer" containerID="ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0"
Jul 21 18:28:36.572030 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.571996    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:28:36.572115 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.572079    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:28:36.572158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:36.572117    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:28:37.580704 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:37.580668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" event={"ID":"0507fee8-e007-4365-a255-3e513dddef9f","Type":"ContainerStarted","Data":"3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275"}
Jul 21 18:28:37.771662 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:37.771626    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:37.771973 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:37.771950    2578 patch_prober.go:28] interesting pod/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 21 18:28:37.772155 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:37.772129    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 21 18:28:38.772179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:38.772131    2578 patch_prober.go:28] interesting pod/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 21 18:28:38.772702 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:38.772221    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 21 18:28:39.771734 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:39.771686    2578 patch_prober.go:28] interesting pod/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 21 18:28:39.771944 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:39.771763    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 21 18:28:40.771908 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:40.771864    2578 patch_prober.go:28] interesting pod/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Jul 21 18:28:40.772396 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:40.771950    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Jul 21 18:28:41.775914 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:41.775879    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:41.776279 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:41.776247    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:41.777045 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:41.777029    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:28:57.560252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.560217    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"]
Jul 21 18:28:57.560743 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.560453    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv" podUID="d6ef9960-a71b-453e-96b6-fd9438fb1331" containerName="maas-api" containerID="cri-o://b8458925139094011db49c967c143cfa16bb7b99f665195aa0778b6cde71b175" gracePeriod=30
Jul 21 18:28:57.685329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.685297    2578 generic.go:356] "Generic (PLEG): container finished" podID="d6ef9960-a71b-453e-96b6-fd9438fb1331" containerID="b8458925139094011db49c967c143cfa16bb7b99f665195aa0778b6cde71b175" exitCode=0
Jul 21 18:28:57.685466 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.685373    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv" event={"ID":"d6ef9960-a71b-453e-96b6-fd9438fb1331","Type":"ContainerDied","Data":"b8458925139094011db49c967c143cfa16bb7b99f665195aa0778b6cde71b175"}
Jul 21 18:28:57.815181 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.815111    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:57.871292 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.871262    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7c9kv\" (UniqueName: \"kubernetes.io/projected/d6ef9960-a71b-453e-96b6-fd9438fb1331-kube-api-access-7c9kv\") pod \"d6ef9960-a71b-453e-96b6-fd9438fb1331\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") "
Jul 21 18:28:57.871447 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.871330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ef9960-a71b-453e-96b6-fd9438fb1331-maas-api-tls\") pod \"d6ef9960-a71b-453e-96b6-fd9438fb1331\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") "
Jul 21 18:28:57.871447 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.871391    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ef9960-a71b-453e-96b6-fd9438fb1331-openshift-service-ca\") pod \"d6ef9960-a71b-453e-96b6-fd9438fb1331\" (UID: \"d6ef9960-a71b-453e-96b6-fd9438fb1331\") "
Jul 21 18:28:57.871763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.871738    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6ef9960-a71b-453e-96b6-fd9438fb1331-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d6ef9960-a71b-453e-96b6-fd9438fb1331" (UID: "d6ef9960-a71b-453e-96b6-fd9438fb1331"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:28:57.873426 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.873405    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6ef9960-a71b-453e-96b6-fd9438fb1331-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d6ef9960-a71b-453e-96b6-fd9438fb1331" (UID: "d6ef9960-a71b-453e-96b6-fd9438fb1331"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:28:57.873490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.873464    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ef9960-a71b-453e-96b6-fd9438fb1331-kube-api-access-7c9kv" (OuterVolumeSpecName: "kube-api-access-7c9kv") pod "d6ef9960-a71b-453e-96b6-fd9438fb1331" (UID: "d6ef9960-a71b-453e-96b6-fd9438fb1331"). InnerVolumeSpecName "kube-api-access-7c9kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:28:57.972815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.972784    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6ef9960-a71b-453e-96b6-fd9438fb1331-openshift-service-ca\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:57.972815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.972809    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7c9kv\" (UniqueName: \"kubernetes.io/projected/d6ef9960-a71b-453e-96b6-fd9438fb1331-kube-api-access-7c9kv\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:57.972815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:57.972819    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6ef9960-a71b-453e-96b6-fd9438fb1331-maas-api-tls\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:28:58.690944 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:58.690910    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv" event={"ID":"d6ef9960-a71b-453e-96b6-fd9438fb1331","Type":"ContainerDied","Data":"01568cb211dcbf1e5b54ccfa4984a2902da58108c4a2ecc2019a68e623ff5190"}
Jul 21 18:28:58.691390 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:58.690967    2578 scope.go:117] "RemoveContainer" containerID="b8458925139094011db49c967c143cfa16bb7b99f665195aa0778b6cde71b175"
Jul 21 18:28:58.691390 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:58.690920    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"
Jul 21 18:28:58.712105 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:58.712082    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"]
Jul 21 18:28:58.715469 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:58.715446    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-ee15a55f-6cf8687774-wt6jv"]
Jul 21 18:28:59.102258 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:28:59.102178    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6ef9960-a71b-453e-96b6-fd9438fb1331" path="/var/lib/kubelet/pods/d6ef9960-a71b-453e-96b6-fd9438fb1331/volumes"
Jul 21 18:29:02.249075 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:02.249042    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"]
Jul 21 18:29:02.249531 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:02.249293    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy" containerID="cri-o://3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275" gracePeriod=30
Jul 21 18:29:03.017653 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.017618    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"]
Jul 21 18:29:03.018076 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018057    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy"
Jul 21 18:29:03.018076 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018079    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy"
Jul 21 18:29:03.018244 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018112    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6ef9960-a71b-453e-96b6-fd9438fb1331" containerName="maas-api"
Jul 21 18:29:03.018244 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018120    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6ef9960-a71b-453e-96b6-fd9438fb1331" containerName="maas-api"
Jul 21 18:29:03.018244 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018140    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy"
Jul 21 18:29:03.018244 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018151    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy"
Jul 21 18:29:03.018445 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018247    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy"
Jul 21 18:29:03.018445 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018262    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6ef9960-a71b-453e-96b6-fd9438fb1331" containerName="maas-api"
Jul 21 18:29:03.018445 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.018280    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="15d771c0-a1f7-441d-bbe0-8e3b5432396f" containerName="istio-proxy"
Jul 21 18:29:03.022922 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.022896    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.025672 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.025585    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-b6157a9f-openshift-default-dockercfg-wv6g4\""
Jul 21 18:29:03.035976 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.035951    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"]
Jul 21 18:29:03.121666 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121618    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-podinfo\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.121834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-socket\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.121834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121730    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd6cd1-061e-4fd0-8be9-865a91831655-istiod-ca-cert\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.121834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121760    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-certs\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.121834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-credential-socket\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.121834 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121815    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-envoy\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.122102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121870    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-token\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.122102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121899    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkjhx\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-kube-api-access-zkjhx\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.122102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.121927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-data\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223284 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-token\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223452 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkjhx\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-kube-api-access-zkjhx\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223557 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223532    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-data\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223717 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223701    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-podinfo\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-socket\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223872 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223827    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd6cd1-061e-4fd0-8be9-865a91831655-istiod-ca-cert\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223871    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-certs\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-credential-socket\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223998 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223930    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-envoy\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.223998 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.223941    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-data\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.224091 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.224072    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-socket\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.224304 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.224274    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-certs\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.224608 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.224569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-credential-socket\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.224888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.224863    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd6cd1-061e-4fd0-8be9-865a91831655-istiod-ca-cert\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.226079 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.226059    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-envoy\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.226466 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.226448    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-podinfo\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.231701 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.231673    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-token\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.231942 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.231921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkjhx\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-kube-api-access-zkjhx\") pod \"e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") " pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.333579 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.333479    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:03.478973 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.478947    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"]
Jul 21 18:29:03.480639 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:29:03.480613    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ffd6cd1_061e_4fd0_8be9_865a91831655.slice/crio-6ce965c5d99dfaf5932e8119e64c82a6e529c210b79713d518906926e9d0f781 WatchSource:0}: Error finding container 6ce965c5d99dfaf5932e8119e64c82a6e529c210b79713d518906926e9d0f781: Status 404 returned error can't find the container with id 6ce965c5d99dfaf5932e8119e64c82a6e529c210b79713d518906926e9d0f781
Jul 21 18:29:03.482528 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.482496    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:03.482674 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.482560    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:03.482674 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.482614    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:03.717714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.717674    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" event={"ID":"5ffd6cd1-061e-4fd0-8be9-865a91831655","Type":"ContainerStarted","Data":"7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020"}
Jul 21 18:29:03.717910 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.717739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" event={"ID":"5ffd6cd1-061e-4fd0-8be9-865a91831655","Type":"ContainerStarted","Data":"6ce965c5d99dfaf5932e8119e64c82a6e529c210b79713d518906926e9d0f781"}
Jul 21 18:29:03.739078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:03.739025    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" podStartSLOduration=1.739011372 podStartE2EDuration="1.739011372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:29:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:29:03.736731747 +0000 UTC m=+1731.995174927" watchObservedRunningTime="2026-07-21 18:29:03.739011372 +0000 UTC m=+1731.997454552"
Jul 21 18:29:04.333814 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:04.333780    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:04.338708 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:04.338685    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:04.722451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:04.722419    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:04.723455 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:04.723432    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:05.197994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.197965    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"]
Jul 21 18:29:05.201342 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.201324    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.208714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.208637    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-b6157a9f-serving-cert\""
Jul 21 18:29:05.222383 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.222354    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"]
Jul 21 18:29:05.343739 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.343706    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v26fx\" (UniqueName: \"kubernetes.io/projected/49073714-93d6-4bb8-a7dc-635a54bda4be-kube-api-access-v26fx\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.344133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.343742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49073714-93d6-4bb8-a7dc-635a54bda4be-maas-api-tls\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.344133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.343857    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49073714-93d6-4bb8-a7dc-635a54bda4be-openshift-service-ca\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.444803 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.444771    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49073714-93d6-4bb8-a7dc-635a54bda4be-openshift-service-ca\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.444960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.444851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v26fx\" (UniqueName: \"kubernetes.io/projected/49073714-93d6-4bb8-a7dc-635a54bda4be-kube-api-access-v26fx\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.444960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.444877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49073714-93d6-4bb8-a7dc-635a54bda4be-maas-api-tls\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.445442 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.445416    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49073714-93d6-4bb8-a7dc-635a54bda4be-openshift-service-ca\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.447424 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.447399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49073714-93d6-4bb8-a7dc-635a54bda4be-maas-api-tls\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.463430 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.463369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v26fx\" (UniqueName: \"kubernetes.io/projected/49073714-93d6-4bb8-a7dc-635a54bda4be-kube-api-access-v26fx\") pod \"maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.512395 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.512370    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:05.850842 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:05.850815    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"]
Jul 21 18:29:05.852762 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:29:05.852736    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49073714_93d6_4bb8_a7dc_635a54bda4be.slice/crio-093bba90e18edab311b44785d2061781640adec3b15d29e224bcbdfeddea7aab WatchSource:0}: Error finding container 093bba90e18edab311b44785d2061781640adec3b15d29e224bcbdfeddea7aab: Status 404 returned error can't find the container with id 093bba90e18edab311b44785d2061781640adec3b15d29e224bcbdfeddea7aab
Jul 21 18:29:06.733027 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:06.732990    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk" event={"ID":"49073714-93d6-4bb8-a7dc-635a54bda4be","Type":"ContainerStarted","Data":"97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73"}
Jul 21 18:29:06.733376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:06.733056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk" event={"ID":"49073714-93d6-4bb8-a7dc-635a54bda4be","Type":"ContainerStarted","Data":"093bba90e18edab311b44785d2061781640adec3b15d29e224bcbdfeddea7aab"}
Jul 21 18:29:06.733376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:06.733217    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:06.749958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:06.749911    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk" podStartSLOduration=1.312563289 podStartE2EDuration="1.749893984s" totalImagesPullingTime="437.330695ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:29:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:29:06.749442563 +0000 UTC m=+1735.007885746" watchObservedRunningTime="2026-07-21 18:29:06.749893984 +0000 UTC m=+1735.008337166"
Jul 21 18:29:07.495418 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.495390    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:29:07.564893 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.564823    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-socket\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.564911    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-credential-socket\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.564936    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-envoy\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.564962    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-istio-token\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565007    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-certs\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565264 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565063    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-data\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565264 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565088    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0507fee8-e007-4365-a255-3e513dddef9f-istio-podinfo\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565264 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565090    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:07.565264 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565118    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4p9ct\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-kube-api-access-4p9ct\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565264 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565214    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:07.565504 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565301    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0507fee8-e007-4365-a255-3e513dddef9f-istiod-ca-cert\") pod \"0507fee8-e007-4365-a255-3e513dddef9f\" (UID: \"0507fee8-e007-4365-a255-3e513dddef9f\") "
Jul 21 18:29:07.565732 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565706    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.565846 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565734    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.565846 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565284    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:07.565846 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565677    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0507fee8-e007-4365-a255-3e513dddef9f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:29:07.566028 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.565952    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-data" (OuterVolumeSpecName: "istio-data") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:07.567208 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.567178    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0507fee8-e007-4365-a255-3e513dddef9f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:29:07.567295 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.567216    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-istio-token" (OuterVolumeSpecName: "istio-token") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:29:07.567295 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.567226    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:07.567295 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.567238    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-kube-api-access-4p9ct" (OuterVolumeSpecName: "kube-api-access-4p9ct") pod "0507fee8-e007-4365-a255-3e513dddef9f" (UID: "0507fee8-e007-4365-a255-3e513dddef9f"). InnerVolumeSpecName "kube-api-access-4p9ct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:29:07.666556 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.666524    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4p9ct\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-kube-api-access-4p9ct\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.666556 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.666549    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0507fee8-e007-4365-a255-3e513dddef9f-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.666556 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.666558    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.666556 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.666566    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0507fee8-e007-4365-a255-3e513dddef9f-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.666841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.666575    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.666841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.666583    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0507fee8-e007-4365-a255-3e513dddef9f-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.666841 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.666613    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0507fee8-e007-4365-a255-3e513dddef9f-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:07.739877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.739841    2578 generic.go:356] "Generic (PLEG): container finished" podID="0507fee8-e007-4365-a255-3e513dddef9f" containerID="3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275" exitCode=0
Jul 21 18:29:07.740242 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.739891    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"
Jul 21 18:29:07.740242 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.739924    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" event={"ID":"0507fee8-e007-4365-a255-3e513dddef9f","Type":"ContainerDied","Data":"3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275"}
Jul 21 18:29:07.740242 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.739986    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4" event={"ID":"0507fee8-e007-4365-a255-3e513dddef9f","Type":"ContainerDied","Data":"7d2074492f4e5167656aed68f5f82e1029a90810b7349efff39b55a03cdee29b"}
Jul 21 18:29:07.740242 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.740018    2578 scope.go:117] "RemoveContainer" containerID="3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275"
Jul 21 18:29:07.749754 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.749737    2578 scope.go:117] "RemoveContainer" containerID="ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0"
Jul 21 18:29:07.757511 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.757495    2578 scope.go:117] "RemoveContainer" containerID="3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275"
Jul 21 18:29:07.757762 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:29:07.757745    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275\": container with ID starting with 3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275 not found: ID does not exist" containerID="3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275"
Jul 21 18:29:07.757822 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.757770    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275"} err="failed to get container status \"3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275\": rpc error: code = NotFound desc = could not find container \"3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275\": container with ID starting with 3da5eefe93f1a087363eadc70f6aba6cc047db01526093b985bc0686851c1275 not found: ID does not exist"
Jul 21 18:29:07.757822 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.757787    2578 scope.go:117] "RemoveContainer" containerID="ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0"
Jul 21 18:29:07.757999 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:29:07.757985    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0\": container with ID starting with ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0 not found: ID does not exist" containerID="ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0"
Jul 21 18:29:07.758038 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.758002    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0"} err="failed to get container status \"ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0\": rpc error: code = NotFound desc = could not find container \"ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0\": container with ID starting with ce0ce3a59cee3bb0677eae32f6971408e09f67b89f19f9ba9c31ce86d616fec0 not found: ID does not exist"
Jul 21 18:29:07.769106 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.769085    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"]
Jul 21 18:29:07.775423 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:07.775401    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-ee15a55f-gw-openshift-default-b6d9b7775-97wb4"]
Jul 21 18:29:09.102421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:09.102386    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0507fee8-e007-4365-a255-3e513dddef9f" path="/var/lib/kubelet/pods/0507fee8-e007-4365-a255-3e513dddef9f/volumes"
Jul 21 18:29:12.745697 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:12.745668    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:19.898747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:19.898712    2578 patch_prober.go:28] interesting pod/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:47100->10.133.0.52:15021: read: connection reset by peer" start-of-body=
Jul 21 18:29:19.899117 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:19.898793    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": read tcp 10.133.0.2:47100->10.133.0.52:15021: read: connection reset by peer"
Jul 21 18:29:20.813438 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:20.813405    2578 generic.go:356] "Generic (PLEG): container finished" podID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerID="7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020" exitCode=0
Jul 21 18:29:20.813633 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:20.813480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" event={"ID":"5ffd6cd1-061e-4fd0-8be9-865a91831655","Type":"ContainerDied","Data":"7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020"}
Jul 21 18:29:20.813883 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:20.813868    2578 scope.go:117] "RemoveContainer" containerID="7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020"
Jul 21 18:29:20.814886 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:20.814860    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:20.814938 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:20.814924    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:20.814974 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:20.814958    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:21.820038 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:21.819998    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" event={"ID":"5ffd6cd1-061e-4fd0-8be9-865a91831655","Type":"ContainerStarted","Data":"6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26"}
Jul 21 18:29:22.334182 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:22.334145    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:22.334378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:22.334354    2578 patch_prober.go:28] interesting pod/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 21 18:29:22.334424 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:22.334399    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 21 18:29:23.334653 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:23.334613    2578 patch_prober.go:28] interesting pod/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 21 18:29:23.335089 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:23.334693    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 21 18:29:24.334819 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:24.334783    2578 patch_prober.go:28] interesting pod/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 21 18:29:24.335219 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:24.334853    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 21 18:29:25.334020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:25.333983    2578 patch_prober.go:28] interesting pod/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 21 18:29:25.334202 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:25.334035    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 21 18:29:26.337869 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:26.337837    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:26.338340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:26.338246    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:26.339045 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:26.339025    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:41.300690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.300652    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"]
Jul 21 18:29:41.301237 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.300931    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk" podUID="49073714-93d6-4bb8-a7dc-635a54bda4be" containerName="maas-api" containerID="cri-o://97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73" gracePeriod=30
Jul 21 18:29:41.553478 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.553423    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:41.684006 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.683971    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49073714-93d6-4bb8-a7dc-635a54bda4be-maas-api-tls\") pod \"49073714-93d6-4bb8-a7dc-635a54bda4be\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") "
Jul 21 18:29:41.684152 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.684041    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49073714-93d6-4bb8-a7dc-635a54bda4be-openshift-service-ca\") pod \"49073714-93d6-4bb8-a7dc-635a54bda4be\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") "
Jul 21 18:29:41.684208 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.684162    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v26fx\" (UniqueName: \"kubernetes.io/projected/49073714-93d6-4bb8-a7dc-635a54bda4be-kube-api-access-v26fx\") pod \"49073714-93d6-4bb8-a7dc-635a54bda4be\" (UID: \"49073714-93d6-4bb8-a7dc-635a54bda4be\") "
Jul 21 18:29:41.684361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.684339    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49073714-93d6-4bb8-a7dc-635a54bda4be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "49073714-93d6-4bb8-a7dc-635a54bda4be" (UID: "49073714-93d6-4bb8-a7dc-635a54bda4be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:29:41.684441 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.684427    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49073714-93d6-4bb8-a7dc-635a54bda4be-openshift-service-ca\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:41.686108 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.686078    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49073714-93d6-4bb8-a7dc-635a54bda4be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "49073714-93d6-4bb8-a7dc-635a54bda4be" (UID: "49073714-93d6-4bb8-a7dc-635a54bda4be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:29:41.686233 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.686178    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49073714-93d6-4bb8-a7dc-635a54bda4be-kube-api-access-v26fx" (OuterVolumeSpecName: "kube-api-access-v26fx") pod "49073714-93d6-4bb8-a7dc-635a54bda4be" (UID: "49073714-93d6-4bb8-a7dc-635a54bda4be"). InnerVolumeSpecName "kube-api-access-v26fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:29:41.785812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.785779    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v26fx\" (UniqueName: \"kubernetes.io/projected/49073714-93d6-4bb8-a7dc-635a54bda4be-kube-api-access-v26fx\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:41.785812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.785808    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49073714-93d6-4bb8-a7dc-635a54bda4be-maas-api-tls\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:41.912555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.912524    2578 generic.go:356] "Generic (PLEG): container finished" podID="49073714-93d6-4bb8-a7dc-635a54bda4be" containerID="97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73" exitCode=0
Jul 21 18:29:41.912747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.912566    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"
Jul 21 18:29:41.912747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.912622    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk" event={"ID":"49073714-93d6-4bb8-a7dc-635a54bda4be","Type":"ContainerDied","Data":"97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73"}
Jul 21 18:29:41.912747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.912681    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk" event={"ID":"49073714-93d6-4bb8-a7dc-635a54bda4be","Type":"ContainerDied","Data":"093bba90e18edab311b44785d2061781640adec3b15d29e224bcbdfeddea7aab"}
Jul 21 18:29:41.912747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.912709    2578 scope.go:117] "RemoveContainer" containerID="97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73"
Jul 21 18:29:41.921491 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.921472    2578 scope.go:117] "RemoveContainer" containerID="97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73"
Jul 21 18:29:41.921773 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:29:41.921747    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73\": container with ID starting with 97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73 not found: ID does not exist" containerID="97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73"
Jul 21 18:29:41.921851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.921785    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73"} err="failed to get container status \"97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73\": rpc error: code = NotFound desc = could not find container \"97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73\": container with ID starting with 97e801f8789a2dab3f8226c8e1c2aeb3e5e463c0b92f871e1f0089f872deea73 not found: ID does not exist"
Jul 21 18:29:41.935106 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.935082    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"]
Jul 21 18:29:41.937469 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:41.937450    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-b6157a9f-7696fcf9fc-8x2fk"]
Jul 21 18:29:43.101530 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:43.101492    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49073714-93d6-4bb8-a7dc-635a54bda4be" path="/var/lib/kubelet/pods/49073714-93d6-4bb8-a7dc-635a54bda4be/volumes"
Jul 21 18:29:47.144527 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:47.144489    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"]
Jul 21 18:29:47.144946 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:47.144773    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy" containerID="cri-o://6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26" gracePeriod=30
Jul 21 18:29:48.046461 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.046424    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"]
Jul 21 18:29:48.046888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.046871    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy"
Jul 21 18:29:48.046888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.046889    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy"
Jul 21 18:29:48.047004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.046908    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy"
Jul 21 18:29:48.047004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.046914    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy"
Jul 21 18:29:48.047004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.046931    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49073714-93d6-4bb8-a7dc-635a54bda4be" containerName="maas-api"
Jul 21 18:29:48.047004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.046938    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="49073714-93d6-4bb8-a7dc-635a54bda4be" containerName="maas-api"
Jul 21 18:29:48.047148 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.047011    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy"
Jul 21 18:29:48.047148 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.047020    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="49073714-93d6-4bb8-a7dc-635a54bda4be" containerName="maas-api"
Jul 21 18:29:48.047148 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.047028    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0507fee8-e007-4365-a255-3e513dddef9f" containerName="istio-proxy"
Jul 21 18:29:48.048912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.048889    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.051717 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.051694    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-7f620e25-openshift-default-dockercfg-glxx8\""
Jul 21 18:29:48.066557 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.066533    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"]
Jul 21 18:29:48.142135 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142101    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-envoy\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142331 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142177    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-data\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142331 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-socket\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142331 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142248    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-certs\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142331 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfgzd\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-kube-api-access-mfgzd\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142546 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4405aa09-b06d-4641-a299-f200fcaa28a1-istiod-ca-cert\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142546 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-podinfo\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142546 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-credential-socket\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.142546 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.142462    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-token\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.243709 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.243661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-data\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.243709 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.243713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-socket\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.243730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-certs\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.243753    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfgzd\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-kube-api-access-mfgzd\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.243787    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4405aa09-b06d-4641-a299-f200fcaa28a1-istiod-ca-cert\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.243804    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-podinfo\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.243824    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-credential-socket\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.244035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-token\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.244099    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-envoy\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.244134    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-data\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244702 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.244309    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-socket\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244702 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.244335    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-credential-socket\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244702 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.244369    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-certs\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.244873 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.244704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4405aa09-b06d-4641-a299-f200fcaa28a1-istiod-ca-cert\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.246256 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.246234    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-envoy\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.246578 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.246556    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-podinfo\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.253876 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.253855    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-token\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.254094 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.254067    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfgzd\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-kube-api-access-mfgzd\") pod \"e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") " pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.358753 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.358671    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:48.496143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.496115    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"]
Jul 21 18:29:48.497316 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:29:48.497287    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4405aa09_b06d_4641_a299_f200fcaa28a1.slice/crio-46e97a472d7574cee384b2a802fc154e19cfe112a4e7e5208282d82ada15d3d3 WatchSource:0}: Error finding container 46e97a472d7574cee384b2a802fc154e19cfe112a4e7e5208282d82ada15d3d3: Status 404 returned error can't find the container with id 46e97a472d7574cee384b2a802fc154e19cfe112a4e7e5208282d82ada15d3d3
Jul 21 18:29:48.499496 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.499463    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:48.499581 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.499530    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:48.499581 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.499562    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:29:48.950290 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.950245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" event={"ID":"4405aa09-b06d-4641-a299-f200fcaa28a1","Type":"ContainerStarted","Data":"bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27"}
Jul 21 18:29:48.950454 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.950317    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" event={"ID":"4405aa09-b06d-4641-a299-f200fcaa28a1","Type":"ContainerStarted","Data":"46e97a472d7574cee384b2a802fc154e19cfe112a4e7e5208282d82ada15d3d3"}
Jul 21 18:29:48.973110 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:48.973060    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" podStartSLOduration=1.973044884 podStartE2EDuration="1.973044884s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:29:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:29:48.972916389 +0000 UTC m=+1777.231359570" watchObservedRunningTime="2026-07-21 18:29:48.973044884 +0000 UTC m=+1777.231488063"
Jul 21 18:29:49.359270 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:49.359185    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:49.364078 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:49.364054    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:49.955118 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:49.955088    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:49.956171 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:49.956152    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:29:50.538139 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.538106    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"]
Jul 21 18:29:50.541010 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.540991    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.543863 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.543838    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-7f620e25-serving-cert\""
Jul 21 18:29:50.556211 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.556190    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"]
Jul 21 18:29:50.665395 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.665356    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f7569-44bc-45fd-a83e-5e5384bd8ca3-openshift-service-ca\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.665577 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.665470    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6fsz\" (UniqueName: \"kubernetes.io/projected/677f7569-44bc-45fd-a83e-5e5384bd8ca3-kube-api-access-p6fsz\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.665577 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.665569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.766469 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.766426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f7569-44bc-45fd-a83e-5e5384bd8ca3-openshift-service-ca\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.766677 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.766583    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6fsz\" (UniqueName: \"kubernetes.io/projected/677f7569-44bc-45fd-a83e-5e5384bd8ca3-kube-api-access-p6fsz\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.766743 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.766675    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.766800 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:29:50.766786    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-serving-cert: secret "maas-api-e2e-derive-7f620e25-serving-cert" not found
Jul 21 18:29:50.766863 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:29:50.766852    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls podName:677f7569-44bc-45fd-a83e-5e5384bd8ca3 nodeName:}" failed. No retries permitted until 2026-07-21 18:29:51.266833576 +0000 UTC m=+1779.525276748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls") pod "maas-api-e2e-derive-7f620e25-56d865c446-7qz4v" (UID: "677f7569-44bc-45fd-a83e-5e5384bd8ca3") : secret "maas-api-e2e-derive-7f620e25-serving-cert" not found
Jul 21 18:29:50.767060 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.767041    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f7569-44bc-45fd-a83e-5e5384bd8ca3-openshift-service-ca\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:50.774694 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:50.774663    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6fsz\" (UniqueName: \"kubernetes.io/projected/677f7569-44bc-45fd-a83e-5e5384bd8ca3-kube-api-access-p6fsz\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:51.272023 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:51.271989    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:51.274247 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:51.274215    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls\") pod \"maas-api-e2e-derive-7f620e25-56d865c446-7qz4v\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:51.452268 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:51.452237    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:51.590227 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:51.590172    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"]
Jul 21 18:29:51.592510 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:29:51.592484    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod677f7569_44bc_45fd_a83e_5e5384bd8ca3.slice/crio-27324718f98854574f13cbf9265352390b71a0b11a232d5681c73eaa7770c577 WatchSource:0}: Error finding container 27324718f98854574f13cbf9265352390b71a0b11a232d5681c73eaa7770c577: Status 404 returned error can't find the container with id 27324718f98854574f13cbf9265352390b71a0b11a232d5681c73eaa7770c577
Jul 21 18:29:51.965975 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:51.965939    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v" event={"ID":"677f7569-44bc-45fd-a83e-5e5384bd8ca3","Type":"ContainerStarted","Data":"27324718f98854574f13cbf9265352390b71a0b11a232d5681c73eaa7770c577"}
Jul 21 18:29:52.396086 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.396058    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:52.483967 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.483934    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-socket\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.483983    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-data\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484002    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-credential-socket\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484028    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-token\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484107    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd6cd1-061e-4fd0-8be9-865a91831655-istiod-ca-cert\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484159    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkjhx\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-kube-api-access-zkjhx\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-envoy\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484282    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:52.484328 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484294    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:52.484539 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484339    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-certs\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484539 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484372    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-podinfo\") pod \"5ffd6cd1-061e-4fd0-8be9-865a91831655\" (UID: \"5ffd6cd1-061e-4fd0-8be9-865a91831655\") "
Jul 21 18:29:52.484688 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5ffd6cd1-061e-4fd0-8be9-865a91831655-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:29:52.484688 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484575    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:52.484815 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484795    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.484870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484820    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.484870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484820    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-data" (OuterVolumeSpecName: "istio-data") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:52.484870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484834    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.484870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.484849    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ffd6cd1-061e-4fd0-8be9-865a91831655-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.486441 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.486410    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-kube-api-access-zkjhx" (OuterVolumeSpecName: "kube-api-access-zkjhx") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "kube-api-access-zkjhx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:29:52.486558 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.486443    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-token" (OuterVolumeSpecName: "istio-token") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:29:52.486558 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.486457    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:29:52.486558 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.486513    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5ffd6cd1-061e-4fd0-8be9-865a91831655" (UID: "5ffd6cd1-061e-4fd0-8be9-865a91831655"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:29:52.586274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.586170    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.586274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.586202    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.586274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.586213    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.586274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.586228    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkjhx\" (UniqueName: \"kubernetes.io/projected/5ffd6cd1-061e-4fd0-8be9-865a91831655-kube-api-access-zkjhx\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.586274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.586241    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ffd6cd1-061e-4fd0-8be9-865a91831655-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:29:52.972274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.972236    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v" event={"ID":"677f7569-44bc-45fd-a83e-5e5384bd8ca3","Type":"ContainerStarted","Data":"0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f"}
Jul 21 18:29:52.972717 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.972347    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:29:52.973770 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.973745    2578 generic.go:356] "Generic (PLEG): container finished" podID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerID="6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26" exitCode=0
Jul 21 18:29:52.973871 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.973790    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"
Jul 21 18:29:52.973871 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.973827    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" event={"ID":"5ffd6cd1-061e-4fd0-8be9-865a91831655","Type":"ContainerDied","Data":"6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26"}
Jul 21 18:29:52.973940 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.973873    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj" event={"ID":"5ffd6cd1-061e-4fd0-8be9-865a91831655","Type":"ContainerDied","Data":"6ce965c5d99dfaf5932e8119e64c82a6e529c210b79713d518906926e9d0f781"}
Jul 21 18:29:52.973940 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.973896    2578 scope.go:117] "RemoveContainer" containerID="6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26"
Jul 21 18:29:52.983401 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.983383    2578 scope.go:117] "RemoveContainer" containerID="7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020"
Jul 21 18:29:52.990607 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.990549    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v" podStartSLOduration=2.593274899 podStartE2EDuration="2.990535492s" totalImagesPullingTime="397.260593ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:29:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:29:52.988983701 +0000 UTC m=+1781.247426885" watchObservedRunningTime="2026-07-21 18:29:52.990535492 +0000 UTC m=+1781.248978718"
Jul 21 18:29:52.991660 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.991644    2578 scope.go:117] "RemoveContainer" containerID="6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26"
Jul 21 18:29:52.991915 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:29:52.991895    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26\": container with ID starting with 6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26 not found: ID does not exist" containerID="6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26"
Jul 21 18:29:52.991991 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.991927    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26"} err="failed to get container status \"6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26\": rpc error: code = NotFound desc = could not find container \"6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26\": container with ID starting with 6c4bb5c77f417a0c940a4c36d567700d3c257d89f8c97c3c0fd72f8f60544d26 not found: ID does not exist"
Jul 21 18:29:52.991991 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.991950    2578 scope.go:117] "RemoveContainer" containerID="7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020"
Jul 21 18:29:52.992191 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:29:52.992173    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020\": container with ID starting with 7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020 not found: ID does not exist" containerID="7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020"
Jul 21 18:29:52.992233 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:52.992197    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020"} err="failed to get container status \"7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020\": rpc error: code = NotFound desc = could not find container \"7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020\": container with ID starting with 7a18616b836c06f9124c4edd508356789ece64d6896969d2e6516311b0946020 not found: ID does not exist"
Jul 21 18:29:53.003409 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:53.003380    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"]
Jul 21 18:29:53.008635 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:53.008608    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-b6157a9f-openshift-default-7d97c96d69-tsntj"]
Jul 21 18:29:53.109364 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:53.109329    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" path="/var/lib/kubelet/pods/5ffd6cd1-061e-4fd0-8be9-865a91831655/volumes"
Jul 21 18:29:58.990848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:29:58.990814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:30:04.956513 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:04.956474    2578 patch_prober.go:28] interesting pod/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 21 18:30:04.957031 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:04.956551    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 21 18:30:05.033434 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:05.033400    2578 generic.go:356] "Generic (PLEG): container finished" podID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerID="bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27" exitCode=0
Jul 21 18:30:05.033638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:05.033443    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" event={"ID":"4405aa09-b06d-4641-a299-f200fcaa28a1","Type":"ContainerDied","Data":"bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27"}
Jul 21 18:30:05.033886 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:05.033871    2578 scope.go:117] "RemoveContainer" containerID="bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27"
Jul 21 18:30:05.034835 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:05.034807    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:30:05.034979 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:05.034880    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:30:05.034979 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:05.034916    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:30:06.040381 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:06.040344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" event={"ID":"4405aa09-b06d-4641-a299-f200fcaa28a1","Type":"ContainerStarted","Data":"c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632"}
Jul 21 18:30:06.358917 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:06.358816    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:30:06.359102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:06.359064    2578 patch_prober.go:28] interesting pod/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 21 18:30:06.359183 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:06.359126    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 21 18:30:07.359294 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:07.359251    2578 patch_prober.go:28] interesting pod/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 21 18:30:07.359746 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:07.359313    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 21 18:30:08.359477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:08.359443    2578 patch_prober.go:28] interesting pod/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 21 18:30:08.359891 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:08.359500    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 21 18:30:09.359571 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:09.359539    2578 patch_prober.go:28] interesting pod/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Jul 21 18:30:09.359983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:09.359611    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Jul 21 18:30:10.363015 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:10.362978    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:30:10.363480 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:10.363366    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:30:10.363958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:10.363936    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:30:13.546186 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:13.546161    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:30:13.548223 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:13.548197    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:30:25.609341 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:25.609308    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"]
Jul 21 18:30:25.609853 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:25.609567    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v" podUID="677f7569-44bc-45fd-a83e-5e5384bd8ca3" containerName="maas-api" containerID="cri-o://0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f" gracePeriod=30
Jul 21 18:30:25.862670 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:25.862582    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:30:26.000718 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.000666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6fsz\" (UniqueName: \"kubernetes.io/projected/677f7569-44bc-45fd-a83e-5e5384bd8ca3-kube-api-access-p6fsz\") pod \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") "
Jul 21 18:30:26.000895 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.000819    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls\") pod \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") "
Jul 21 18:30:26.000895 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.000852    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f7569-44bc-45fd-a83e-5e5384bd8ca3-openshift-service-ca\") pod \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\" (UID: \"677f7569-44bc-45fd-a83e-5e5384bd8ca3\") "
Jul 21 18:30:26.001368 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.001331    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/677f7569-44bc-45fd-a83e-5e5384bd8ca3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "677f7569-44bc-45fd-a83e-5e5384bd8ca3" (UID: "677f7569-44bc-45fd-a83e-5e5384bd8ca3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:30:26.002840 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.002817    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/677f7569-44bc-45fd-a83e-5e5384bd8ca3-kube-api-access-p6fsz" (OuterVolumeSpecName: "kube-api-access-p6fsz") pod "677f7569-44bc-45fd-a83e-5e5384bd8ca3" (UID: "677f7569-44bc-45fd-a83e-5e5384bd8ca3"). InnerVolumeSpecName "kube-api-access-p6fsz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:30:26.002923 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.002894    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "677f7569-44bc-45fd-a83e-5e5384bd8ca3" (UID: "677f7569-44bc-45fd-a83e-5e5384bd8ca3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:30:26.102085 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.102048    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/677f7569-44bc-45fd-a83e-5e5384bd8ca3-maas-api-tls\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:26.102085 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.102079    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/677f7569-44bc-45fd-a83e-5e5384bd8ca3-openshift-service-ca\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:26.102085 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.102089    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6fsz\" (UniqueName: \"kubernetes.io/projected/677f7569-44bc-45fd-a83e-5e5384bd8ca3-kube-api-access-p6fsz\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:26.144321 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.144289    2578 generic.go:356] "Generic (PLEG): container finished" podID="677f7569-44bc-45fd-a83e-5e5384bd8ca3" containerID="0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f" exitCode=0
Jul 21 18:30:26.144483 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.144337    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"
Jul 21 18:30:26.144483 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.144368    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v" event={"ID":"677f7569-44bc-45fd-a83e-5e5384bd8ca3","Type":"ContainerDied","Data":"0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f"}
Jul 21 18:30:26.144483 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.144421    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v" event={"ID":"677f7569-44bc-45fd-a83e-5e5384bd8ca3","Type":"ContainerDied","Data":"27324718f98854574f13cbf9265352390b71a0b11a232d5681c73eaa7770c577"}
Jul 21 18:30:26.144483 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.144442    2578 scope.go:117] "RemoveContainer" containerID="0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f"
Jul 21 18:30:26.153930 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.153915    2578 scope.go:117] "RemoveContainer" containerID="0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f"
Jul 21 18:30:26.154178 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:30:26.154157    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f\": container with ID starting with 0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f not found: ID does not exist" containerID="0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f"
Jul 21 18:30:26.154230 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.154185    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f"} err="failed to get container status \"0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f\": rpc error: code = NotFound desc = could not find container \"0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f\": container with ID starting with 0795ad7fe6490a4cb4034b9ad288f432624b75eab230652c08851d4b6c3c9f5f not found: ID does not exist"
Jul 21 18:30:26.166164 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.166139    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"]
Jul 21 18:30:26.172418 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:26.172396    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-7f620e25-56d865c446-7qz4v"]
Jul 21 18:30:27.101788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:27.101752    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="677f7569-44bc-45fd-a83e-5e5384bd8ca3" path="/var/lib/kubelet/pods/677f7569-44bc-45fd-a83e-5e5384bd8ca3/volumes"
Jul 21 18:30:30.398338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:30.398306    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"]
Jul 21 18:30:30.398821 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:30.398525    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy" containerID="cri-o://c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632" gracePeriod=30
Jul 21 18:30:35.659953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.659880    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:30:35.795323 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795288    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfgzd\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-kube-api-access-mfgzd\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795330    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-token\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795372    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-data\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795398    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-certs\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795437    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-socket\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795500 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795472    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4405aa09-b06d-4641-a299-f200fcaa28a1-istiod-ca-cert\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795584    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-credential-socket\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795662    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-podinfo\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795693    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-envoy\") pod \"4405aa09-b06d-4641-a299-f200fcaa28a1\" (UID: \"4405aa09-b06d-4641-a299-f200fcaa28a1\") "
Jul 21 18:30:35.795940 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795844    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:30:35.796170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.795985    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:30:35.796170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.796005    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4405aa09-b06d-4641-a299-f200fcaa28a1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:30:35.796170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.796096    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.796170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.796114    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.796170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.796128    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4405aa09-b06d-4641-a299-f200fcaa28a1-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.796170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.796141    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:30:35.796539 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.796403    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-data" (OuterVolumeSpecName: "istio-data") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:30:35.797768 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.797747    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-token" (OuterVolumeSpecName: "istio-token") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:30:35.797988 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.797958    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:30:35.798094 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.798028    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:30:35.798094 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.798069    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-kube-api-access-mfgzd" (OuterVolumeSpecName: "kube-api-access-mfgzd") pod "4405aa09-b06d-4641-a299-f200fcaa28a1" (UID: "4405aa09-b06d-4641-a299-f200fcaa28a1"). InnerVolumeSpecName "kube-api-access-mfgzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:30:35.896928 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.896889    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.896928 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.896919    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.896928 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.896931    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfgzd\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-kube-api-access-mfgzd\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.897170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.896943    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.897170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.896955    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:35.897170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:35.896965    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4405aa09-b06d-4641-a299-f200fcaa28a1-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:30:36.191999 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.191959    2578 generic.go:356] "Generic (PLEG): container finished" podID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerID="c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632" exitCode=0
Jul 21 18:30:36.192201 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.192018    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"
Jul 21 18:30:36.192201 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.192050    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" event={"ID":"4405aa09-b06d-4641-a299-f200fcaa28a1","Type":"ContainerDied","Data":"c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632"}
Jul 21 18:30:36.192201 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.192119    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2" event={"ID":"4405aa09-b06d-4641-a299-f200fcaa28a1","Type":"ContainerDied","Data":"46e97a472d7574cee384b2a802fc154e19cfe112a4e7e5208282d82ada15d3d3"}
Jul 21 18:30:36.192201 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.192153    2578 scope.go:117] "RemoveContainer" containerID="c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632"
Jul 21 18:30:36.201921 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.201901    2578 scope.go:117] "RemoveContainer" containerID="bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27"
Jul 21 18:30:36.212887 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.212860    2578 scope.go:117] "RemoveContainer" containerID="c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632"
Jul 21 18:30:36.213374 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:30:36.213354    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632\": container with ID starting with c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632 not found: ID does not exist" containerID="c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632"
Jul 21 18:30:36.213467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.213383    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632"} err="failed to get container status \"c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632\": rpc error: code = NotFound desc = could not find container \"c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632\": container with ID starting with c4019908faa720e4ca3a5b5c987b1d0fb03b610ecf5af36cf9a447253abc0632 not found: ID does not exist"
Jul 21 18:30:36.213467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.213402    2578 scope.go:117] "RemoveContainer" containerID="bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27"
Jul 21 18:30:36.213671 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:30:36.213655    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27\": container with ID starting with bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27 not found: ID does not exist" containerID="bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27"
Jul 21 18:30:36.213726 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.213677    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27"} err="failed to get container status \"bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27\": rpc error: code = NotFound desc = could not find container \"bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27\": container with ID starting with bb2f06ae87acb9f9e28d58cf4be7e96e24c949b571d5c329e0643037b1863d27 not found: ID does not exist"
Jul 21 18:30:36.214927 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.214908    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"]
Jul 21 18:30:36.218438 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:36.218409    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-7f620e25-openshift-default-7df68ff57f-h4ls2"]
Jul 21 18:30:37.101447 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:30:37.101414    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" path="/var/lib/kubelet/pods/4405aa09-b06d-4641-a299-f200fcaa28a1/volumes"
Jul 21 18:32:12.082263 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082181    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"]
Jul 21 18:32:12.082762 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082746    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy"
Jul 21 18:32:12.082816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082765    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy"
Jul 21 18:32:12.082816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082782    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="677f7569-44bc-45fd-a83e-5e5384bd8ca3" containerName="maas-api"
Jul 21 18:32:12.082816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082792    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="677f7569-44bc-45fd-a83e-5e5384bd8ca3" containerName="maas-api"
Jul 21 18:32:12.082816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082806    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy"
Jul 21 18:32:12.082816 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082816    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy"
Jul 21 18:32:12.082967 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082830    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy"
Jul 21 18:32:12.082967 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082839    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy"
Jul 21 18:32:12.082967 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082854    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy"
Jul 21 18:32:12.082967 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082863    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy"
Jul 21 18:32:12.082967 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082950    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy"
Jul 21 18:32:12.082967 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082961    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="677f7569-44bc-45fd-a83e-5e5384bd8ca3" containerName="maas-api"
Jul 21 18:32:12.083141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082972    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy"
Jul 21 18:32:12.083141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082982    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4405aa09-b06d-4641-a299-f200fcaa28a1" containerName="istio-proxy"
Jul 21 18:32:12.083141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.082991    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ffd6cd1-061e-4fd0-8be9-865a91831655" containerName="istio-proxy"
Jul 21 18:32:12.085598 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.085570    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.088608 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.088564    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-26807d71-openshift-default-dockercfg-lm5tq\""
Jul 21 18:32:12.104889 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.104866    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"]
Jul 21 18:32:12.243289 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-envoy\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243304    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-socket\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243323    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wcs94\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-kube-api-access-wcs94\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243386    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-credential-socket\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243421    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-token\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243463    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-podinfo\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243718 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243569    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-certs\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243718 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243708    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-data\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.243821 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.243735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11da283c-66b9-4af6-8d73-568eedf39bd6-istiod-ca-cert\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344546    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-data\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11da283c-66b9-4af6-8d73-568eedf39bd6-istiod-ca-cert\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-envoy\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-socket\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344695    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wcs94\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-kube-api-access-wcs94\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344805    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-credential-socket\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344838    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-token\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.344956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.344868    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-podinfo\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.345208 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.345026    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-data\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.345208 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.345073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-socket\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.345208 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.345145    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-certs\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.345453 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.345205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-credential-socket\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.345518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.345451    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-certs\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.345677 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.345645    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11da283c-66b9-4af6-8d73-568eedf39bd6-istiod-ca-cert\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.347229 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.347207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-envoy\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.347354 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.347337    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-podinfo\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.360022 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.359995    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-token\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.360111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.360096    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wcs94\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-kube-api-access-wcs94\") pod \"e2e-mt-26807d71-openshift-default-66df59697f-nl4hq\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") " pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.396206 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.396173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:12.531262 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.531236    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"]
Jul 21 18:32:12.533345 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:32:12.533323    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11da283c_66b9_4af6_8d73_568eedf39bd6.slice/crio-97ca37ef3268a29c56506940ba95ad367014c42d2862e96a8344288a3ddf02b9 WatchSource:0}: Error finding container 97ca37ef3268a29c56506940ba95ad367014c42d2862e96a8344288a3ddf02b9: Status 404 returned error can't find the container with id 97ca37ef3268a29c56506940ba95ad367014c42d2862e96a8344288a3ddf02b9
Jul 21 18:32:12.535440 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.535406    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:32:12.535544 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.535482    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:32:12.535544 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.535529    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:32:12.624177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.624109    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" event={"ID":"11da283c-66b9-4af6-8d73-568eedf39bd6","Type":"ContainerStarted","Data":"f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca"}
Jul 21 18:32:12.624177 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.624161    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" event={"ID":"11da283c-66b9-4af6-8d73-568eedf39bd6","Type":"ContainerStarted","Data":"97ca37ef3268a29c56506940ba95ad367014c42d2862e96a8344288a3ddf02b9"}
Jul 21 18:32:12.646912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:12.646860    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" podStartSLOduration=0.646844092 podStartE2EDuration="646.844092ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:32:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:32:12.645420924 +0000 UTC m=+1920.903864102" watchObservedRunningTime="2026-07-21 18:32:12.646844092 +0000 UTC m=+1920.905287271"
Jul 21 18:32:13.397324 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:13.397281    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:13.402220 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:13.402199    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:13.628186 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:13.628155    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:13.629179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:13.629159    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:26.691150 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:26.691113    2578 generic.go:356] "Generic (PLEG): container finished" podID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerID="f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca" exitCode=0
Jul 21 18:32:26.691571 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:26.691189    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" event={"ID":"11da283c-66b9-4af6-8d73-568eedf39bd6","Type":"ContainerDied","Data":"f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca"}
Jul 21 18:32:26.691706 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:26.691689    2578 scope.go:117] "RemoveContainer" containerID="f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca"
Jul 21 18:32:26.692769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:26.692739    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:32:26.692837 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:26.692807    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:32:26.692877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:26.692841    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:32:27.706848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:27.706809    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" event={"ID":"11da283c-66b9-4af6-8d73-568eedf39bd6","Type":"ContainerStarted","Data":"6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311"}
Jul 21 18:32:28.397066 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:28.397033    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:28.398293 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:28.398262    2578 patch_prober.go:28] interesting pod/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 21 18:32:28.398351 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:28.398328    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 21 18:32:28.628948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:28.628914    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:29.396645 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:29.396579    2578 patch_prober.go:28] interesting pod/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 21 18:32:29.397111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:29.396680    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 21 18:32:30.397184 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:30.397143    2578 patch_prober.go:28] interesting pod/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 21 18:32:30.397656 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:30.397213    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 21 18:32:31.396892 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:31.396852    2578 patch_prober.go:28] interesting pod/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 21 18:32:31.397108 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:31.396936    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 21 18:32:32.400794 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:32.400760    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:32.401765 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:32.401747    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:48.532774 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:48.532731    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"]
Jul 21 18:32:48.533278 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:48.533009    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy" containerID="cri-o://6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311" gracePeriod=30
Jul 21 18:32:53.785584 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.785556    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:53.838300 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.838209    2578 generic.go:356] "Generic (PLEG): container finished" podID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerID="6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311" exitCode=0
Jul 21 18:32:53.838300 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.838262    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"
Jul 21 18:32:53.838480 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.838295    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" event={"ID":"11da283c-66b9-4af6-8d73-568eedf39bd6","Type":"ContainerDied","Data":"6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311"}
Jul 21 18:32:53.838480 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.838354    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq" event={"ID":"11da283c-66b9-4af6-8d73-568eedf39bd6","Type":"ContainerDied","Data":"97ca37ef3268a29c56506940ba95ad367014c42d2862e96a8344288a3ddf02b9"}
Jul 21 18:32:53.838480 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.838384    2578 scope.go:117] "RemoveContainer" containerID="6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311"
Jul 21 18:32:53.846971 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.846952    2578 scope.go:117] "RemoveContainer" containerID="f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca"
Jul 21 18:32:53.854770 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.854752    2578 scope.go:117] "RemoveContainer" containerID="6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311"
Jul 21 18:32:53.855059 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:32:53.855040    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311\": container with ID starting with 6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311 not found: ID does not exist" containerID="6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311"
Jul 21 18:32:53.855111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.855069    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311"} err="failed to get container status \"6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311\": rpc error: code = NotFound desc = could not find container \"6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311\": container with ID starting with 6deba45d04fe9658eac362b5fb52c6fd694559c7f11ebcf6c6f000e93aa7a311 not found: ID does not exist"
Jul 21 18:32:53.855111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.855089    2578 scope.go:117] "RemoveContainer" containerID="f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca"
Jul 21 18:32:53.855328 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:32:53.855311    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca\": container with ID starting with f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca not found: ID does not exist" containerID="f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca"
Jul 21 18:32:53.855378 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.855337    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca"} err="failed to get container status \"f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca\": rpc error: code = NotFound desc = could not find container \"f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca\": container with ID starting with f1e5300e9b511a9a842430d52df98071c78382699a8292e2e82a061e1ec659ca not found: ID does not exist"
Jul 21 18:32:53.916004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.915979    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-envoy\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916017    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-credential-socket\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916057    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-token\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916082    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-data\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916106    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-podinfo\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916123    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-certs\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916392 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916160    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-socket\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916392 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916201    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wcs94\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-kube-api-access-wcs94\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916392 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916285    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11da283c-66b9-4af6-8d73-568eedf39bd6-istiod-ca-cert\") pod \"11da283c-66b9-4af6-8d73-568eedf39bd6\" (UID: \"11da283c-66b9-4af6-8d73-568eedf39bd6\") "
Jul 21 18:32:53.916563 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916436    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:32:53.916563 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916447    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:32:53.916706 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916560    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:32:53.916706 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916620    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:53.916706 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916644    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:53.916872 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.916746    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11da283c-66b9-4af6-8d73-568eedf39bd6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:32:53.917207 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.917187    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-data" (OuterVolumeSpecName: "istio-data") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:32:53.918237 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.918211    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-token" (OuterVolumeSpecName: "istio-token") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:32:53.918507 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.918478    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-kube-api-access-wcs94" (OuterVolumeSpecName: "kube-api-access-wcs94") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "kube-api-access-wcs94". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:32:53.918620 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.918538    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:32:53.918688 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:53.918643    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "11da283c-66b9-4af6-8d73-568eedf39bd6" (UID: "11da283c-66b9-4af6-8d73-568eedf39bd6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:32:54.017267 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.017232    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:54.017267 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.017259    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:54.017267 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.017271    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:54.017490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.017282    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:54.017490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.017295    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wcs94\" (UniqueName: \"kubernetes.io/projected/11da283c-66b9-4af6-8d73-568eedf39bd6-kube-api-access-wcs94\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:54.017490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.017307    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11da283c-66b9-4af6-8d73-568eedf39bd6-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:54.017490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.017318    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11da283c-66b9-4af6-8d73-568eedf39bd6-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:32:54.160391 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.160360    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"]
Jul 21 18:32:54.165922 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:54.165898    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-26807d71-openshift-default-66df59697f-nl4hq"]
Jul 21 18:32:55.101918 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:32:55.101881    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" path="/var/lib/kubelet/pods/11da283c-66b9-4af6-8d73-568eedf39bd6/volumes"
Jul 21 18:33:08.842532 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.842450    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"]
Jul 21 18:33:08.843030 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.842818    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy"
Jul 21 18:33:08.843030 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.842829    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy"
Jul 21 18:33:08.843030 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.842892    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy"
Jul 21 18:33:08.843030 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.842901    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy"
Jul 21 18:33:08.845817 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.845798    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.849283 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.849261    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-15fecbf7-openshift-default-dockercfg-dvp42\""
Jul 21 18:33:08.868880 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.868856    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"]
Jul 21 18:33:08.952638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952582    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-socket\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.952820 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-certs\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.952820 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-envoy\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.952820 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-credential-socket\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.952820 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952755    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-data\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.952820 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952778    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24xw2\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-kube-api-access-24xw2\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.952820 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952817    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-podinfo\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.953166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952867    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-token\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:08.953166 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:08.952938    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istiod-ca-cert\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.053871 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.053834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-credential-socket\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054040 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.053884    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-data\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054040 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.053912    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24xw2\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-kube-api-access-24xw2\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054040 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.053947    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-podinfo\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054040 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.053969    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-token\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054248 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054116    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istiod-ca-cert\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054248 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054200    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-socket\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054367 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-certs\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054367 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054288    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-envoy\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054367 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-data\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054529 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054361    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-credential-socket\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054587 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-socket\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054687 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054639    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-certs\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.054881 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.054859    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istiod-ca-cert\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.056346 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.056326    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-envoy\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.056542 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.056523    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-podinfo\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.062477 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.062453    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-token\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.063029 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.063012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24xw2\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-kube-api-access-24xw2\") pod \"e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") " pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.155647 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.155616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:09.286853 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.286821    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"]
Jul 21 18:33:09.289956 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:33:09.289931    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod292b7026_04f4_40e9_ba3c_49cb2cbb6f33.slice/crio-a50e56ebc56e21a1295285f80836bda4b307e8ca112ec988f54d67b72c1c00af WatchSource:0}: Error finding container a50e56ebc56e21a1295285f80836bda4b307e8ca112ec988f54d67b72c1c00af: Status 404 returned error can't find the container with id a50e56ebc56e21a1295285f80836bda4b307e8ca112ec988f54d67b72c1c00af
Jul 21 18:33:09.292031 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.291995    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:33:09.292141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.292077    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:33:09.292141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.292127    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:33:09.912045 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.912010    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" event={"ID":"292b7026-04f4-40e9-ba3c-49cb2cbb6f33","Type":"ContainerStarted","Data":"4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151"}
Jul 21 18:33:09.912494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.912060    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" event={"ID":"292b7026-04f4-40e9-ba3c-49cb2cbb6f33","Type":"ContainerStarted","Data":"a50e56ebc56e21a1295285f80836bda4b307e8ca112ec988f54d67b72c1c00af"}
Jul 21 18:33:09.936611 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:09.934301    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" podStartSLOduration=1.934280928 podStartE2EDuration="1.934280928s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:33:09.933711784 +0000 UTC m=+1978.192154965" watchObservedRunningTime="2026-07-21 18:33:09.934280928 +0000 UTC m=+1978.192724087"
Jul 21 18:33:10.156055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:10.156023    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:10.160909 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:10.160882    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:10.916785 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:10.916755    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:10.917670 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:10.917649    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:26.530231 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.529905    2578 patch_prober.go:28] interesting pod/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:39744->10.133.0.57:15021: read: connection reset by peer" start-of-body=
Jul 21 18:33:26.530231 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.529988    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:39744->10.133.0.57:15021: read: connection reset by peer"
Jul 21 18:33:26.993614 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.993569    2578 generic.go:356] "Generic (PLEG): container finished" podID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerID="4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151" exitCode=0
Jul 21 18:33:26.993786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.993623    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" event={"ID":"292b7026-04f4-40e9-ba3c-49cb2cbb6f33","Type":"ContainerDied","Data":"4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151"}
Jul 21 18:33:26.994048 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.994033    2578 scope.go:117] "RemoveContainer" containerID="4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151"
Jul 21 18:33:26.995044 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.995014    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:33:26.995111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.995086    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:33:26.995148 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:26.995134    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:33:28.000252 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:28.000212    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" event={"ID":"292b7026-04f4-40e9-ba3c-49cb2cbb6f33","Type":"ContainerStarted","Data":"d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129"}
Jul 21 18:33:28.156140 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:28.156106    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:28.156358 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:28.156330    2578 patch_prober.go:28] interesting pod/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 21 18:33:28.156439 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:28.156388    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 21 18:33:29.156506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:29.156468    2578 patch_prober.go:28] interesting pod/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 21 18:33:29.156912 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:29.156525    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 21 18:33:30.156318 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:30.156281    2578 patch_prober.go:28] interesting pod/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 21 18:33:30.156519 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:30.156338    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 21 18:33:31.156739 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:31.156704    2578 patch_prober.go:28] interesting pod/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 21 18:33:31.157113 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:31.156759    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 21 18:33:32.160156 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:32.160126    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:32.160581 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:32.160387    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:32.161095 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:32.161080    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:33:33.622777 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.622745    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"]
Jul 21 18:33:33.623182 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.623123    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy"
Jul 21 18:33:33.623182 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.623134    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="11da283c-66b9-4af6-8d73-568eedf39bd6" containerName="istio-proxy"
Jul 21 18:33:33.625570 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.625546    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.630436 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.630413    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-15fecbf7\"/\"openshift-service-ca.crt\""
Jul 21 18:33:33.630577 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.630502    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-15fecbf7\"/\"default-dockercfg-rg7jd\""
Jul 21 18:33:33.630577 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.630550    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-15fecbf7\"/\"kube-root-ca.crt\""
Jul 21 18:33:33.630694 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.630649    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-15fecbf7\"/\"e2e-lifecycle-model-15fecbf7-kserve-self-signed-certs\""
Jul 21 18:33:33.638665 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.638401    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"]
Jul 21 18:33:33.698994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.698960    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-model-cache\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.698994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.698995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tmp-dir\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.699210 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.699023    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x5gh\" (UniqueName: \"kubernetes.io/projected/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-kube-api-access-4x5gh\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.699210 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.699116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-home\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.699210 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.699149    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tls-certs\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.699210 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.699170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-dshm\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.800459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.800423    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tmp-dir\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.800459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.800475    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x5gh\" (UniqueName: \"kubernetes.io/projected/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-kube-api-access-4x5gh\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.800752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.800531    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-home\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.800752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.800554    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tls-certs\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.800752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.800578    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-dshm\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.800752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.800678    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-model-cache\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.801363 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.801139    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-model-cache\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.801929 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.801901    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-home\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.802139 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.802108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tmp-dir\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.803963 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.803937    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-dshm\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.804124 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.804105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tls-certs\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.810227 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.810203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x5gh\" (UniqueName: \"kubernetes.io/projected/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-kube-api-access-4x5gh\") pod \"e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") " pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:33.936093 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:33.936046    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:34.076805 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:34.076780    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"]
Jul 21 18:33:34.078877 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:33:34.078852    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32f5aede_595a_4b9d_a4da_cb1e8baadb0b.slice/crio-662ac99e9210099aa5dd895161e34d4acf17d6f1fa694d95a90e978d9a2012e4 WatchSource:0}: Error finding container 662ac99e9210099aa5dd895161e34d4acf17d6f1fa694d95a90e978d9a2012e4: Status 404 returned error can't find the container with id 662ac99e9210099aa5dd895161e34d4acf17d6f1fa694d95a90e978d9a2012e4
Jul 21 18:33:35.041001 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:35.040965    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql" event={"ID":"32f5aede-595a-4b9d-a4da-cb1e8baadb0b","Type":"ContainerStarted","Data":"71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c"}
Jul 21 18:33:35.041001 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:35.041005    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql" event={"ID":"32f5aede-595a-4b9d-a4da-cb1e8baadb0b","Type":"ContainerStarted","Data":"662ac99e9210099aa5dd895161e34d4acf17d6f1fa694d95a90e978d9a2012e4"}
Jul 21 18:33:35.060221 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:35.060170    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql" podStartSLOduration=2.060155609 podStartE2EDuration="2.060155609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:33:35.058582623 +0000 UTC m=+2003.317025804" watchObservedRunningTime="2026-07-21 18:33:35.060155609 +0000 UTC m=+2003.318598788"
Jul 21 18:33:43.936546 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:43.936513    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:43.936973 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:43.936569    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:43.949125 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:43.949100    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:33:44.106079 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:33:44.106048    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:34:22.219858 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:22.219815    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"]
Jul 21 18:34:37.240291 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.240200    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql" podUID="32f5aede-595a-4b9d-a4da-cb1e8baadb0b" containerName="main" containerID="cri-o://71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c" gracePeriod=45
Jul 21 18:34:37.494481 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.494421    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:34:37.577172 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577141    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-dshm\") pod \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") "
Jul 21 18:34:37.577337 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577183    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tmp-dir\") pod \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") "
Jul 21 18:34:37.577337 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577211    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-model-cache\") pod \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") "
Jul 21 18:34:37.577337 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577237    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tls-certs\") pod \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") "
Jul 21 18:34:37.577337 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577274    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4x5gh\" (UniqueName: \"kubernetes.io/projected/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-kube-api-access-4x5gh\") pod \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") "
Jul 21 18:34:37.577337 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577317    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-home\") pod \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\" (UID: \"32f5aede-595a-4b9d-a4da-cb1e8baadb0b\") "
Jul 21 18:34:37.577628 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577427    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "32f5aede-595a-4b9d-a4da-cb1e8baadb0b" (UID: "32f5aede-595a-4b9d-a4da-cb1e8baadb0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:34:37.577628 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577459    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "32f5aede-595a-4b9d-a4da-cb1e8baadb0b" (UID: "32f5aede-595a-4b9d-a4da-cb1e8baadb0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:34:37.577628 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577616    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tmp-dir\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:34:37.577777 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577634    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-model-cache\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:34:37.577777 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.577645    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-home" (OuterVolumeSpecName: "home") pod "32f5aede-595a-4b9d-a4da-cb1e8baadb0b" (UID: "32f5aede-595a-4b9d-a4da-cb1e8baadb0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:34:37.579426 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.579404    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-dshm" (OuterVolumeSpecName: "dshm") pod "32f5aede-595a-4b9d-a4da-cb1e8baadb0b" (UID: "32f5aede-595a-4b9d-a4da-cb1e8baadb0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:34:37.579508 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.579474    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "32f5aede-595a-4b9d-a4da-cb1e8baadb0b" (UID: "32f5aede-595a-4b9d-a4da-cb1e8baadb0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:34:37.579508 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.579476    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-kube-api-access-4x5gh" (OuterVolumeSpecName: "kube-api-access-4x5gh") pod "32f5aede-595a-4b9d-a4da-cb1e8baadb0b" (UID: "32f5aede-595a-4b9d-a4da-cb1e8baadb0b"). InnerVolumeSpecName "kube-api-access-4x5gh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:34:37.678667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.678630    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-dshm\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:34:37.678667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.678664    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-tls-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:34:37.678667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.678676    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4x5gh\" (UniqueName: \"kubernetes.io/projected/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-kube-api-access-4x5gh\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:34:37.678881 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:37.678686    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32f5aede-595a-4b9d-a4da-cb1e8baadb0b-home\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:34:38.348622 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.348571    2578 generic.go:356] "Generic (PLEG): container finished" podID="32f5aede-595a-4b9d-a4da-cb1e8baadb0b" containerID="71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c" exitCode=0
Jul 21 18:34:38.349025 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.348636    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"
Jul 21 18:34:38.349025 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.348661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql" event={"ID":"32f5aede-595a-4b9d-a4da-cb1e8baadb0b","Type":"ContainerDied","Data":"71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c"}
Jul 21 18:34:38.349025 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.348707    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql" event={"ID":"32f5aede-595a-4b9d-a4da-cb1e8baadb0b","Type":"ContainerDied","Data":"662ac99e9210099aa5dd895161e34d4acf17d6f1fa694d95a90e978d9a2012e4"}
Jul 21 18:34:38.349025 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.348725    2578 scope.go:117] "RemoveContainer" containerID="71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c"
Jul 21 18:34:38.357556 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.357537    2578 scope.go:117] "RemoveContainer" containerID="71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c"
Jul 21 18:34:38.357882 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:34:38.357863    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c\": container with ID starting with 71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c not found: ID does not exist" containerID="71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c"
Jul 21 18:34:38.357943 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.357888    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c"} err="failed to get container status \"71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c\": rpc error: code = NotFound desc = could not find container \"71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c\": container with ID starting with 71a47151339330403ddcaaa210f8241d8a6c0c607ed08cbdbf8a21192e16d96c not found: ID does not exist"
Jul 21 18:34:38.370638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.370617    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"]
Jul 21 18:34:38.374619 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:38.374586    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-15fecbf7/e2e-lifecycle-model-15fecbf7-kserve-67c68b4ddc-q4dql"]
Jul 21 18:34:39.102643 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:34:39.102584    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32f5aede-595a-4b9d-a4da-cb1e8baadb0b" path="/var/lib/kubelet/pods/32f5aede-595a-4b9d-a4da-cb1e8baadb0b/volumes"
Jul 21 18:35:05.540164 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:05.540127    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"]
Jul 21 18:35:05.540700 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:05.540372    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy" containerID="cri-o://d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129" gracePeriod=30
Jul 21 18:35:10.795004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.794978    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:35:10.858802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858728    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-socket\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.858802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858786    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24xw2\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-kube-api-access-24xw2\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-data\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858833    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-envoy\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858858    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-certs\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858877    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-token\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858925    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-podinfo\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858964    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istiod-ca-cert\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.858993    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-credential-socket\") pod \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\" (UID: \"292b7026-04f4-40e9-ba3c-49cb2cbb6f33\") "
Jul 21 18:35:10.859383 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.859028    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:35:10.859383 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.859212    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:35:10.859383 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.859284    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:35:10.859383 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.859292    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.859619 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.859520    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:35:10.859768 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.859745    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-data" (OuterVolumeSpecName: "istio-data") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:35:10.861254 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.861219    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-kube-api-access-24xw2" (OuterVolumeSpecName: "kube-api-access-24xw2") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "kube-api-access-24xw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:35:10.861368 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.861315    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:35:10.861474 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.861452    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:35:10.861556 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.861533    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-token" (OuterVolumeSpecName: "istio-token") pod "292b7026-04f4-40e9-ba3c-49cb2cbb6f33" (UID: "292b7026-04f4-40e9-ba3c-49cb2cbb6f33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:35:10.960607 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960549    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24xw2\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-kube-api-access-24xw2\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.960607 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960606    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.960607 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960617    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.960828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960625    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.960828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960632    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.960828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960640    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.960828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960647    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:10.960828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:10.960654    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/292b7026-04f4-40e9-ba3c-49cb2cbb6f33-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:35:11.506504 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.506465    2578 generic.go:356] "Generic (PLEG): container finished" podID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerID="d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129" exitCode=0
Jul 21 18:35:11.506711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.506520    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"
Jul 21 18:35:11.506711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.506552    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" event={"ID":"292b7026-04f4-40e9-ba3c-49cb2cbb6f33","Type":"ContainerDied","Data":"d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129"}
Jul 21 18:35:11.506711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.506640    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r" event={"ID":"292b7026-04f4-40e9-ba3c-49cb2cbb6f33","Type":"ContainerDied","Data":"a50e56ebc56e21a1295285f80836bda4b307e8ca112ec988f54d67b72c1c00af"}
Jul 21 18:35:11.506711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.506671    2578 scope.go:117] "RemoveContainer" containerID="d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129"
Jul 21 18:35:11.515151 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.515133    2578 scope.go:117] "RemoveContainer" containerID="4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151"
Jul 21 18:35:11.522459 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.522445    2578 scope.go:117] "RemoveContainer" containerID="d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129"
Jul 21 18:35:11.522721 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:35:11.522698    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129\": container with ID starting with d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129 not found: ID does not exist" containerID="d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129"
Jul 21 18:35:11.522768 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.522734    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129"} err="failed to get container status \"d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129\": rpc error: code = NotFound desc = could not find container \"d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129\": container with ID starting with d3d7e21a5998a01c01f3b71a0dc40159c602a2db9571dddba27347a987789129 not found: ID does not exist"
Jul 21 18:35:11.522768 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.522758    2578 scope.go:117] "RemoveContainer" containerID="4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151"
Jul 21 18:35:11.522993 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:35:11.522975    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151\": container with ID starting with 4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151 not found: ID does not exist" containerID="4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151"
Jul 21 18:35:11.523038 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.523003    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151"} err="failed to get container status \"4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151\": rpc error: code = NotFound desc = could not find container \"4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151\": container with ID starting with 4cc06ce51f802244c955b298f65b9d80ae14545f491c974dd0a944e9b7684151 not found: ID does not exist"
Jul 21 18:35:11.531407 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.531386    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"]
Jul 21 18:35:11.536290 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:11.536269    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-15fecbf7-openshift-default-5484d858c-96t8r"]
Jul 21 18:35:13.101878 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:13.101848    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" path="/var/lib/kubelet/pods/292b7026-04f4-40e9-ba3c-49cb2cbb6f33/volumes"
Jul 21 18:35:13.577416 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:13.577384    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:35:13.580953 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:35:13.580926    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:36:01.504825 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.504792    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"]
Jul 21 18:36:01.505281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.505146    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy"
Jul 21 18:36:01.505281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.505158    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy"
Jul 21 18:36:01.505281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.505173    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32f5aede-595a-4b9d-a4da-cb1e8baadb0b" containerName="main"
Jul 21 18:36:01.505281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.505178    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="32f5aede-595a-4b9d-a4da-cb1e8baadb0b" containerName="main"
Jul 21 18:36:01.505281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.505247    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="32f5aede-595a-4b9d-a4da-cb1e8baadb0b" containerName="main"
Jul 21 18:36:01.505281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.505257    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy"
Jul 21 18:36:01.505281 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.505262    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy"
Jul 21 18:36:01.507035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.507017    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.509647 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.509626    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-4d82e2-openshift-default-dockercfg-dmsnz\""
Jul 21 18:36:01.519154 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.519130    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"]
Jul 21 18:36:01.621381 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621340    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-data\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621381 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621381    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-certs\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621664 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-token\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621664 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621481    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmcm6\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-kube-api-access-jmcm6\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621664 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621513    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-socket\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621664 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-credential-socket\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621664 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b65f2602-6918-4e1e-9e24-8e73ac11351c-istiod-ca-cert\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-envoy\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.621958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.621698    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-podinfo\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722408 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722370    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-data\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722408 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-certs\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722431    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-token\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722462    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmcm6\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-kube-api-access-jmcm6\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722481    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-socket\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-credential-socket\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b65f2602-6918-4e1e-9e24-8e73ac11351c-istiod-ca-cert\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722609    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-envoy\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.722714 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-podinfo\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.723098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.722968    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-certs\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.723098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.723047    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-credential-socket\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.723098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.723074    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-data\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.723586 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.723563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-socket\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.724039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.724021    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b65f2602-6918-4e1e-9e24-8e73ac11351c-istiod-ca-cert\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.725255 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.725230    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-envoy\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.725770 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.725747    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-podinfo\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.731444 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.731419    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-token\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.731526 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.731420    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmcm6\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-kube-api-access-jmcm6\") pod \"e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") " pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.817558 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.817460    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:01.964780 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.964753    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"]
Jul 21 18:36:01.966422 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:36:01.966391    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb65f2602_6918_4e1e_9e24_8e73ac11351c.slice/crio-9fed17a1792ec3c275ef6534b73691e55cc2841654e89ac997def04046848a74 WatchSource:0}: Error finding container 9fed17a1792ec3c275ef6534b73691e55cc2841654e89ac997def04046848a74: Status 404 returned error can't find the container with id 9fed17a1792ec3c275ef6534b73691e55cc2841654e89ac997def04046848a74
Jul 21 18:36:01.973247 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.973213    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:01.973332 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.973282    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:01.973332 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:01.973310    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:02.742343 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:02.742311    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" event={"ID":"b65f2602-6918-4e1e-9e24-8e73ac11351c","Type":"ContainerStarted","Data":"bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d"}
Jul 21 18:36:02.742720 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:02.742363    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" event={"ID":"b65f2602-6918-4e1e-9e24-8e73ac11351c","Type":"ContainerStarted","Data":"9fed17a1792ec3c275ef6534b73691e55cc2841654e89ac997def04046848a74"}
Jul 21 18:36:02.768952 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:02.768901    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" podStartSLOduration=1.768884324 podStartE2EDuration="1.768884324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:36:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:36:02.766570466 +0000 UTC m=+2151.025013673" watchObservedRunningTime="2026-07-21 18:36:02.768884324 +0000 UTC m=+2151.027327504"
Jul 21 18:36:02.818413 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:02.818377    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:03.823376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:03.823298    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:04.758473 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:04.758435    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:04.759545 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:04.759513    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:09.846101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.846069    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"]
Jul 21 18:36:09.846569 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.846439    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy"
Jul 21 18:36:09.846569 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.846451    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="292b7026-04f4-40e9-ba3c-49cb2cbb6f33" containerName="istio-proxy"
Jul 21 18:36:09.848294 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.848277    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:09.851409 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.851388    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b2c46d-openshift-default-dockercfg-6nlf5\""
Jul 21 18:36:09.862970 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.862943    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"]
Jul 21 18:36:09.999566 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn862\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-kube-api-access-cn862\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:09.999802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999644    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-certs\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:09.999802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999694    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-data\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:09.999802 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999732    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca675797-babf-46a0-a767-f98191308516-istio-podinfo\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:09.999916 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999801    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-credential-socket\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:09.999916 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999844    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-socket\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:09.999916 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999898    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca675797-babf-46a0-a767-f98191308516-istiod-ca-cert\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.000007 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999925    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-istio-token\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.000007 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:09.999947    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-envoy\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca675797-babf-46a0-a767-f98191308516-istiod-ca-cert\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-istio-token\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101214    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-envoy\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101501 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn862\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-kube-api-access-cn862\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101501 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101465    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-certs\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101669 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101523    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-data\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101669 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101573    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca675797-babf-46a0-a767-f98191308516-istio-podinfo\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101669 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101663    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-credential-socket\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-socket\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101819    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-certs\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.101898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-data\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.102067 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.101925    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca675797-babf-46a0-a767-f98191308516-istiod-ca-cert\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.102067 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.102056    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-socket\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.102164 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.102108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-credential-socket\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.103637 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.103615    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-envoy\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.104110 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.104089    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca675797-babf-46a0-a767-f98191308516-istio-podinfo\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.109984 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.109956    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-istio-token\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.110083 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.110028    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn862\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-kube-api-access-cn862\") pod \"e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") " pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.159124 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.159093    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:10.302859 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.302836    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"]
Jul 21 18:36:10.304991 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:36:10.304953    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca675797_babf_46a0_a767_f98191308516.slice/crio-aee88f76b413499c80d59d30560ceae6df3d746c90ca3943fbc4ed3e00c7965a WatchSource:0}: Error finding container aee88f76b413499c80d59d30560ceae6df3d746c90ca3943fbc4ed3e00c7965a: Status 404 returned error can't find the container with id aee88f76b413499c80d59d30560ceae6df3d746c90ca3943fbc4ed3e00c7965a
Jul 21 18:36:10.307168 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.307139    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:10.307236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.307200    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:10.311325 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.307709    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:10.788606 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.788557    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" event={"ID":"ca675797-babf-46a0-a767-f98191308516","Type":"ContainerStarted","Data":"4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486"}
Jul 21 18:36:10.788785 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.788657    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" event={"ID":"ca675797-babf-46a0-a767-f98191308516","Type":"ContainerStarted","Data":"aee88f76b413499c80d59d30560ceae6df3d746c90ca3943fbc4ed3e00c7965a"}
Jul 21 18:36:10.812876 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:10.812831    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" podStartSLOduration=1.8128158330000002 podStartE2EDuration="1.812815833s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:36:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:36:10.810831854 +0000 UTC m=+2159.069275057" watchObservedRunningTime="2026-07-21 18:36:10.812815833 +0000 UTC m=+2159.071259089"
Jul 21 18:36:11.159657 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:11.159620    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:11.164562 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:11.164535    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:11.794688 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:11.794639    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:11.795677 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:11.795653    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:18.831493 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:18.831452    2578 generic.go:356] "Generic (PLEG): container finished" podID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerID="bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d" exitCode=0
Jul 21 18:36:18.831931 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:18.831527    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" event={"ID":"b65f2602-6918-4e1e-9e24-8e73ac11351c","Type":"ContainerDied","Data":"bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d"}
Jul 21 18:36:18.832020 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:18.832001    2578 scope.go:117] "RemoveContainer" containerID="bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d"
Jul 21 18:36:18.833077 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:18.833045    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:18.833185 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:18.833131    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:18.833253 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:18.833191    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:19.758530 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:19.758489    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:19.837951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:19.837910    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" event={"ID":"b65f2602-6918-4e1e-9e24-8e73ac11351c","Type":"ContainerStarted","Data":"38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6"}
Jul 21 18:36:20.818009 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:20.817945    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:20.818995 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:20.818954    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 18:36:20.819256 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:20.819209    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 18:36:21.818662 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:21.818619    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 18:36:21.819251 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:21.819215    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 18:36:22.818244 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:22.818210    2578 patch_prober.go:28] interesting pod/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Jul 21 18:36:22.818524 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:22.818500    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Jul 21 18:36:23.822480 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:23.822447    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:23.822909 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:23.822784    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:23.823390 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:23.823367    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:36:26.796366 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.796272    2578 patch_prober.go:28] interesting pod/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 21 18:36:26.796366 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.796333    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 21 18:36:26.873683 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.873644    2578 generic.go:356] "Generic (PLEG): container finished" podID="ca675797-babf-46a0-a767-f98191308516" containerID="4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486" exitCode=0
Jul 21 18:36:26.873869 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.873718    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" event={"ID":"ca675797-babf-46a0-a767-f98191308516","Type":"ContainerDied","Data":"4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486"}
Jul 21 18:36:26.874158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.874138    2578 scope.go:117] "RemoveContainer" containerID="4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486"
Jul 21 18:36:26.875277 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.875241    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:26.875402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.875333    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:26.875402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:26.875383    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:36:27.886522 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:27.886472    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" event={"ID":"ca675797-babf-46a0-a767-f98191308516","Type":"ContainerStarted","Data":"5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976"}
Jul 21 18:36:28.160063 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:28.159972    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:28.160275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:28.160248    2578 patch_prober.go:28] interesting pod/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 21 18:36:28.160362 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:28.160323    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 21 18:36:29.160550 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:29.160515    2578 patch_prober.go:28] interesting pod/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 21 18:36:29.160957 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:29.160573    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 21 18:36:30.160110 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:30.160071    2578 patch_prober.go:28] interesting pod/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 21 18:36:30.160286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:30.160162    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 21 18:36:31.160528 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:31.160492    2578 patch_prober.go:28] interesting pod/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Jul 21 18:36:31.160947 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:31.160547    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Jul 21 18:36:32.163489 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:32.163458    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:32.164044 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:32.163799    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:32.164442 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:32.164426    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:36:36.174179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.174145    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"]
Jul 21 18:36:36.176402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.176383    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.182865 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.182843    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4d82e2\"/\"default-dockercfg-q7dpp\""
Jul 21 18:36:36.182995 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.182845    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-4d82e2\"/\"test-model-4d82e2-kserve-self-signed-certs\""
Jul 21 18:36:36.182995 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.182845    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4d82e2\"/\"openshift-service-ca.crt\""
Jul 21 18:36:36.182995 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.182873    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-4d82e2\"/\"kube-root-ca.crt\""
Jul 21 18:36:36.198358 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.198335    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"]
Jul 21 18:36:36.269398 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.269367    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-home\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.269579 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.269407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mjbc\" (UniqueName: \"kubernetes.io/projected/35a64c3a-638a-4f29-b81e-0ded3acea2d7-kube-api-access-2mjbc\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.269661 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.269575    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tmp-dir\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.269703 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.269678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-model-cache\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.269745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.269713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-dshm\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.269745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.269735    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tls-certs\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.370491 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.370449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-home\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.370491 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.370501    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mjbc\" (UniqueName: \"kubernetes.io/projected/35a64c3a-638a-4f29-b81e-0ded3acea2d7-kube-api-access-2mjbc\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.370781 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.370626    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tmp-dir\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.370781 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.370684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-model-cache\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.370781 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.370712    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-dshm\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.370781 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.370734    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tls-certs\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.371977 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.371410    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-home\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.372112 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.372070    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-model-cache\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.372340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.372315    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tmp-dir\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.374039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.374009    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tls-certs\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.374669 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.374644    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-dshm\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.380280 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.380253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mjbc\" (UniqueName: \"kubernetes.io/projected/35a64c3a-638a-4f29-b81e-0ded3acea2d7-kube-api-access-2mjbc\") pod \"test-model-4d82e2-kserve-687859c9d7-hfz46\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") " pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.485889 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.485806    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:36.616642 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.616616    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"]
Jul 21 18:36:36.619188 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:36:36.619158    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35a64c3a_638a_4f29_b81e_0ded3acea2d7.slice/crio-9f9110e499f2997e92d3b1c5a3414a157080225bd118e86135eac10a4cf9a8dc WatchSource:0}: Error finding container 9f9110e499f2997e92d3b1c5a3414a157080225bd118e86135eac10a4cf9a8dc: Status 404 returned error can't find the container with id 9f9110e499f2997e92d3b1c5a3414a157080225bd118e86135eac10a4cf9a8dc
Jul 21 18:36:36.932490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.932461    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46" event={"ID":"35a64c3a-638a-4f29-b81e-0ded3acea2d7","Type":"ContainerStarted","Data":"f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462"}
Jul 21 18:36:36.932687 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.932498    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46" event={"ID":"35a64c3a-638a-4f29-b81e-0ded3acea2d7","Type":"ContainerStarted","Data":"9f9110e499f2997e92d3b1c5a3414a157080225bd118e86135eac10a4cf9a8dc"}
Jul 21 18:36:36.951171 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:36.951120    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46" podStartSLOduration=0.95110078 podStartE2EDuration="951.10078ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:36:36.949983076 +0000 UTC m=+2185.208426255" watchObservedRunningTime="2026-07-21 18:36:36.95110078 +0000 UTC m=+2185.209543961"
Jul 21 18:36:46.486089 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:46.486051    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:46.486543 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:46.486100    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:46.502948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:46.502922    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:36:46.994653 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:36:46.994628    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:37:47.267806 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:37:47.267727    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"]
Jul 21 18:38:02.287799 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.287751    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46" podUID="35a64c3a-638a-4f29-b81e-0ded3acea2d7" containerName="main" containerID="cri-o://f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462" gracePeriod=45
Jul 21 18:38:02.553003 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.552980    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:38:02.664783 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.664746    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-home\") pod \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") "
Jul 21 18:38:02.664978 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.664836    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-dshm\") pod \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") "
Jul 21 18:38:02.664978 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.664878    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tmp-dir\") pod \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") "
Jul 21 18:38:02.664978 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.664939    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tls-certs\") pod \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") "
Jul 21 18:38:02.664978 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.664968    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2mjbc\" (UniqueName: \"kubernetes.io/projected/35a64c3a-638a-4f29-b81e-0ded3acea2d7-kube-api-access-2mjbc\") pod \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") "
Jul 21 18:38:02.665204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.665000    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-model-cache\") pod \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\" (UID: \"35a64c3a-638a-4f29-b81e-0ded3acea2d7\") "
Jul 21 18:38:02.665204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.665074    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-home" (OuterVolumeSpecName: "home") pod "35a64c3a-638a-4f29-b81e-0ded3acea2d7" (UID: "35a64c3a-638a-4f29-b81e-0ded3acea2d7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:02.665204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.665184    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "35a64c3a-638a-4f29-b81e-0ded3acea2d7" (UID: "35a64c3a-638a-4f29-b81e-0ded3acea2d7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:02.665334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.665261    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-model-cache" (OuterVolumeSpecName: "model-cache") pod "35a64c3a-638a-4f29-b81e-0ded3acea2d7" (UID: "35a64c3a-638a-4f29-b81e-0ded3acea2d7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:02.665421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.665395    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-model-cache\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:02.665421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.665422    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-home\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:02.665638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.665438    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tmp-dir\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:02.667121 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.667093    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-dshm" (OuterVolumeSpecName: "dshm") pod "35a64c3a-638a-4f29-b81e-0ded3acea2d7" (UID: "35a64c3a-638a-4f29-b81e-0ded3acea2d7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:02.667121 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.667113    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35a64c3a-638a-4f29-b81e-0ded3acea2d7-kube-api-access-2mjbc" (OuterVolumeSpecName: "kube-api-access-2mjbc") pod "35a64c3a-638a-4f29-b81e-0ded3acea2d7" (UID: "35a64c3a-638a-4f29-b81e-0ded3acea2d7"). InnerVolumeSpecName "kube-api-access-2mjbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:38:02.667273 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.667147    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "35a64c3a-638a-4f29-b81e-0ded3acea2d7" (UID: "35a64c3a-638a-4f29-b81e-0ded3acea2d7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:38:02.766647 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.766587    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/35a64c3a-638a-4f29-b81e-0ded3acea2d7-dshm\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:02.766647 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.766642    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/35a64c3a-638a-4f29-b81e-0ded3acea2d7-tls-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:02.766647 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:02.766655    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2mjbc\" (UniqueName: \"kubernetes.io/projected/35a64c3a-638a-4f29-b81e-0ded3acea2d7-kube-api-access-2mjbc\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:03.339403 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.339370    2578 generic.go:356] "Generic (PLEG): container finished" podID="35a64c3a-638a-4f29-b81e-0ded3acea2d7" containerID="f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462" exitCode=0
Jul 21 18:38:03.339828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.339408    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46" event={"ID":"35a64c3a-638a-4f29-b81e-0ded3acea2d7","Type":"ContainerDied","Data":"f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462"}
Jul 21 18:38:03.339828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.339413    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"
Jul 21 18:38:03.339828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.339439    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46" event={"ID":"35a64c3a-638a-4f29-b81e-0ded3acea2d7","Type":"ContainerDied","Data":"9f9110e499f2997e92d3b1c5a3414a157080225bd118e86135eac10a4cf9a8dc"}
Jul 21 18:38:03.339828 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.339457    2578 scope.go:117] "RemoveContainer" containerID="f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462"
Jul 21 18:38:03.348014 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.347987    2578 scope.go:117] "RemoveContainer" containerID="f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462"
Jul 21 18:38:03.348282 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:38:03.348263    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462\": container with ID starting with f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462 not found: ID does not exist" containerID="f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462"
Jul 21 18:38:03.348345 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.348297    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462"} err="failed to get container status \"f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462\": rpc error: code = NotFound desc = could not find container \"f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462\": container with ID starting with f5845f5a5ad3091de96e7b9028a0b51f4464705dd6fd32c5a5a307b4c62ea462 not found: ID does not exist"
Jul 21 18:38:03.360043 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.359980    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"]
Jul 21 18:38:03.366368 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:03.366344    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-4d82e2/test-model-4d82e2-kserve-687859c9d7-hfz46"]
Jul 21 18:38:05.102372 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:05.102339    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35a64c3a-638a-4f29-b81e-0ded3acea2d7" path="/var/lib/kubelet/pods/35a64c3a-638a-4f29-b81e-0ded3acea2d7/volumes"
Jul 21 18:38:11.721818 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:11.721785    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"]
Jul 21 18:38:11.722238 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:11.721999    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy" containerID="cri-o://38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6" gracePeriod=30
Jul 21 18:38:16.976941 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.976913    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:38:16.999667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999586    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-data\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:16.999667 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999643    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-token\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:16.999862 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999676    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmcm6\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-kube-api-access-jmcm6\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:16.999862 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999704    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-credential-socket\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:16.999862 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999736    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-podinfo\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:16.999862 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999789    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-envoy\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:16.999862 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999825    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-socket\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:17.000120 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999866    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b65f2602-6918-4e1e-9e24-8e73ac11351c-istiod-ca-cert\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:17.000120 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999897    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-certs\") pod \"b65f2602-6918-4e1e-9e24-8e73ac11351c\" (UID: \"b65f2602-6918-4e1e-9e24-8e73ac11351c\") "
Jul 21 18:38:17.000120 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:16.999998    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:17.000274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.000215    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:17.000274 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.000259    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.000367 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.000326    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:17.000551 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.000518    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-data" (OuterVolumeSpecName: "istio-data") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:17.000661 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.000550    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b65f2602-6918-4e1e-9e24-8e73ac11351c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:38:17.002031 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.002003    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-kube-api-access-jmcm6" (OuterVolumeSpecName: "kube-api-access-jmcm6") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "kube-api-access-jmcm6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:38:17.002305 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.002281    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-token" (OuterVolumeSpecName: "istio-token") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:38:17.002426 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.002408    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:38:17.002494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.002458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b65f2602-6918-4e1e-9e24-8e73ac11351c" (UID: "b65f2602-6918-4e1e-9e24-8e73ac11351c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:17.101126 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101093    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.101126 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101124    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.101316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101138    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.101316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101152    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmcm6\" (UniqueName: \"kubernetes.io/projected/b65f2602-6918-4e1e-9e24-8e73ac11351c-kube-api-access-jmcm6\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.101316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101166    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.101316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101180    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.101316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101194    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b65f2602-6918-4e1e-9e24-8e73ac11351c-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.101316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.101206    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b65f2602-6918-4e1e-9e24-8e73ac11351c-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:17.408572 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.408543    2578 generic.go:356] "Generic (PLEG): container finished" podID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerID="38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6" exitCode=0
Jul 21 18:38:17.408745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.408622    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"
Jul 21 18:38:17.408745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.408628    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" event={"ID":"b65f2602-6918-4e1e-9e24-8e73ac11351c","Type":"ContainerDied","Data":"38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6"}
Jul 21 18:38:17.408745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.408685    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9" event={"ID":"b65f2602-6918-4e1e-9e24-8e73ac11351c","Type":"ContainerDied","Data":"9fed17a1792ec3c275ef6534b73691e55cc2841654e89ac997def04046848a74"}
Jul 21 18:38:17.408745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.408707    2578 scope.go:117] "RemoveContainer" containerID="38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6"
Jul 21 18:38:17.417170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.417154    2578 scope.go:117] "RemoveContainer" containerID="bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d"
Jul 21 18:38:17.424972 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.424956    2578 scope.go:117] "RemoveContainer" containerID="38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6"
Jul 21 18:38:17.425211 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:38:17.425193    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6\": container with ID starting with 38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6 not found: ID does not exist" containerID="38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6"
Jul 21 18:38:17.425275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.425218    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6"} err="failed to get container status \"38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6\": rpc error: code = NotFound desc = could not find container \"38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6\": container with ID starting with 38bdf3b8d1474a63c806d6912ed03db501eda6927455e099a669e9d3ae05b7d6 not found: ID does not exist"
Jul 21 18:38:17.425275 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.425237    2578 scope.go:117] "RemoveContainer" containerID="bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d"
Jul 21 18:38:17.425463 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:38:17.425449    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d\": container with ID starting with bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d not found: ID does not exist" containerID="bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d"
Jul 21 18:38:17.425505 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.425468    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d"} err="failed to get container status \"bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d\": rpc error: code = NotFound desc = could not find container \"bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d\": container with ID starting with bd87a5a8ef0980737a66f62c0692a18b096f260c5009de783a8382494214899d not found: ID does not exist"
Jul 21 18:38:17.440271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.440245    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"]
Jul 21 18:38:17.454948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:17.454907    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-4d82e2-openshift-default-66789cf5f9-drxp9"]
Jul 21 18:38:19.102562 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:19.102516    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" path="/var/lib/kubelet/pods/b65f2602-6918-4e1e-9e24-8e73ac11351c/volumes"
Jul 21 18:38:35.981551 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:35.981513    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"]
Jul 21 18:38:35.982009 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:35.981739    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy" containerID="cri-o://5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976" gracePeriod=30
Jul 21 18:38:37.089259 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089224    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"]
Jul 21 18:38:37.089641 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089573    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy"
Jul 21 18:38:37.089641 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089583    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy"
Jul 21 18:38:37.089641 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089608    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35a64c3a-638a-4f29-b81e-0ded3acea2d7" containerName="main"
Jul 21 18:38:37.089641 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089615    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="35a64c3a-638a-4f29-b81e-0ded3acea2d7" containerName="main"
Jul 21 18:38:37.089641 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089627    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy"
Jul 21 18:38:37.089641 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089634    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy"
Jul 21 18:38:37.089845 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089705    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy"
Jul 21 18:38:37.089845 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.089714    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="35a64c3a-638a-4f29-b81e-0ded3acea2d7" containerName="main"
Jul 21 18:38:37.094156 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.094139    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.097982 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.097961    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-9c004f-openshift-default-dockercfg-vffw2\""
Jul 21 18:38:37.174170 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.174125    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"]
Jul 21 18:38:37.175833 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.175802    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-socket\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.175989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.175849    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01158ac3-e6ff-4171-b57d-9ec50024f740-istiod-ca-cert\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.175989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.175906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-certs\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.175989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.175951    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-credential-socket\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.175989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.175979    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-envoy\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.176204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.176028    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-data\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.176204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.176064    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-token\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.176204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.176116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-podinfo\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.176204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.176169    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vxd2\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-kube-api-access-5vxd2\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.276793 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.276752    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vxd2\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-kube-api-access-5vxd2\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.276983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.276832    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-socket\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.276983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.276858    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01158ac3-e6ff-4171-b57d-9ec50024f740-istiod-ca-cert\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.276983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.276891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-certs\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.276983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.276936    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-credential-socket\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.276983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.276959    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-envoy\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.276983 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.276979    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-data\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.277293 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.277005    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-token\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.277293 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.277065    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-podinfo\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.277293 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.277278    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-socket\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.277452 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.277358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-certs\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.277553 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.277502    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-data\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.277735 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.277555    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-credential-socket\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.277803 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.277668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01158ac3-e6ff-4171-b57d-9ec50024f740-istiod-ca-cert\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.279365 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.279345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-envoy\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.279605 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.279573    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-podinfo\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.290725 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.290704    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-token\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.290888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.290859    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vxd2\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-kube-api-access-5vxd2\") pod \"e2e-api-a-9c004f-openshift-default-59987455d7-bfww6\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") " pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.407243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.407202    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:37.583829 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.583800    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"]
Jul 21 18:38:37.585526 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:38:37.585501    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01158ac3_e6ff_4171_b57d_9ec50024f740.slice/crio-300e1705a16ec6372bee65bd4ac8c10fd0842f6535c17a3855412c8b1fb676ab WatchSource:0}: Error finding container 300e1705a16ec6372bee65bd4ac8c10fd0842f6535c17a3855412c8b1fb676ab: Status 404 returned error can't find the container with id 300e1705a16ec6372bee65bd4ac8c10fd0842f6535c17a3855412c8b1fb676ab
Jul 21 18:38:37.587625 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.587572    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:37.587711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.587658    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:37.587711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:37.587691    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:38.523720 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:38.523690    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" event={"ID":"01158ac3-e6ff-4171-b57d-9ec50024f740","Type":"ContainerStarted","Data":"d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46"}
Jul 21 18:38:38.524095 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:38.523739    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" event={"ID":"01158ac3-e6ff-4171-b57d-9ec50024f740","Type":"ContainerStarted","Data":"300e1705a16ec6372bee65bd4ac8c10fd0842f6535c17a3855412c8b1fb676ab"}
Jul 21 18:38:38.579633 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:38.579565    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podStartSLOduration=1.579549216 podStartE2EDuration="1.579549216s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:38:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:38:38.578561677 +0000 UTC m=+2306.837004857" watchObservedRunningTime="2026-07-21 18:38:38.579549216 +0000 UTC m=+2306.837992396"
Jul 21 18:38:39.408116 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:39.408080    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:39.413285 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:39.413256    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:39.529662 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:39.529626    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:39.530578 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:39.530558    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:41.248511 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.248449    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:38:41.313852 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.313820    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca675797-babf-46a0-a767-f98191308516-istiod-ca-cert\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314037 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.313896    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-istio-token\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314235 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314182    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca675797-babf-46a0-a767-f98191308516-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:38:41.314353 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314235    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-envoy\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314353 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314278    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-data\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314462 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314349    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-credential-socket\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314462 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca675797-babf-46a0-a767-f98191308516-istio-podinfo\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314462 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314450    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-certs\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314643 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314474    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-socket\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.314643 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314507    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cn862\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-kube-api-access-cn862\") pod \"ca675797-babf-46a0-a767-f98191308516\" (UID: \"ca675797-babf-46a0-a767-f98191308516\") "
Jul 21 18:38:41.315429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.314836    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ca675797-babf-46a0-a767-f98191308516-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.315429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.315175    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:41.315429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.315203    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-data" (OuterVolumeSpecName: "istio-data") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:41.315429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.315347    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:41.315429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.315395    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:41.316817 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.316790    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-istio-token" (OuterVolumeSpecName: "istio-token") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:38:41.317432 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.317404    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ca675797-babf-46a0-a767-f98191308516-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:38:41.317676 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.317650    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-kube-api-access-cn862" (OuterVolumeSpecName: "kube-api-access-cn862") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "kube-api-access-cn862". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:38:41.318391 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.318369    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ca675797-babf-46a0-a767-f98191308516" (UID: "ca675797-babf-46a0-a767-f98191308516"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:38:41.415490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415460    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.415490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415490    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.415695 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415503    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.415695 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415516    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.415695 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415530    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ca675797-babf-46a0-a767-f98191308516-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.415695 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415545    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.415695 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415554    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ca675797-babf-46a0-a767-f98191308516-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.415695 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.415563    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn862\" (UniqueName: \"kubernetes.io/projected/ca675797-babf-46a0-a767-f98191308516-kube-api-access-cn862\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:38:41.542080 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.542000    2578 generic.go:356] "Generic (PLEG): container finished" podID="ca675797-babf-46a0-a767-f98191308516" containerID="5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976" exitCode=0
Jul 21 18:38:41.542080 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.542052    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"
Jul 21 18:38:41.542284 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.542087    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" event={"ID":"ca675797-babf-46a0-a767-f98191308516","Type":"ContainerDied","Data":"5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976"}
Jul 21 18:38:41.542284 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.542150    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d" event={"ID":"ca675797-babf-46a0-a767-f98191308516","Type":"ContainerDied","Data":"aee88f76b413499c80d59d30560ceae6df3d746c90ca3943fbc4ed3e00c7965a"}
Jul 21 18:38:41.542284 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.542183    2578 scope.go:117] "RemoveContainer" containerID="5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976"
Jul 21 18:38:41.551419 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.551403    2578 scope.go:117] "RemoveContainer" containerID="4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486"
Jul 21 18:38:41.558987 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.558970    2578 scope.go:117] "RemoveContainer" containerID="5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976"
Jul 21 18:38:41.559211 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:38:41.559194    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976\": container with ID starting with 5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976 not found: ID does not exist" containerID="5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976"
Jul 21 18:38:41.559256 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.559219    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976"} err="failed to get container status \"5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976\": rpc error: code = NotFound desc = could not find container \"5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976\": container with ID starting with 5f115739f715495c312150a78904a3f5e856a1cc85d119d36648a14777aaa976 not found: ID does not exist"
Jul 21 18:38:41.559256 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.559235    2578 scope.go:117] "RemoveContainer" containerID="4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486"
Jul 21 18:38:41.559441 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:38:41.559423    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486\": container with ID starting with 4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486 not found: ID does not exist" containerID="4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486"
Jul 21 18:38:41.559481 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.559448    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486"} err="failed to get container status \"4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486\": rpc error: code = NotFound desc = could not find container \"4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486\": container with ID starting with 4164c1b83184384e9d1721fefe058c9d5922ec661d07d5e376270c1b12950486 not found: ID does not exist"
Jul 21 18:38:41.577375 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.577350    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"]
Jul 21 18:38:41.585739 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:41.585718    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b2c46d-openshift-default-588c85744f-5xh8d"]
Jul 21 18:38:43.101891 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:43.101863    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca675797-babf-46a0-a767-f98191308516" path="/var/lib/kubelet/pods/ca675797-babf-46a0-a767-f98191308516/volumes"
Jul 21 18:38:46.094530 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.094486    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"]
Jul 21 18:38:46.094898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.094878    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy"
Jul 21 18:38:46.094898 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.094892    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy"
Jul 21 18:38:46.094965 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.094959    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy"
Jul 21 18:38:46.095011 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.094966    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy"
Jul 21 18:38:46.095011 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.094975    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b65f2602-6918-4e1e-9e24-8e73ac11351c" containerName="istio-proxy"
Jul 21 18:38:46.099353 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.099333    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.102955 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.102934    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-057ec0-openshift-default-dockercfg-sp9kp\""
Jul 21 18:38:46.122555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.122527    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"]
Jul 21 18:38:46.257395 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257353    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-podinfo\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-socket\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257465    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-envoy\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257503    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-token\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257529    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-certs\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257813 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-data\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257813 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257687    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-credential-socket\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257813 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257710    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istiod-ca-cert\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.257813 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.257725    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kx87v\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-kube-api-access-kx87v\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358549 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358451    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-socket\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358549 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358500    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-envoy\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358549 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358527    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-token\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358554    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-certs\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358627    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-data\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358660    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-credential-socket\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358686    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istiod-ca-cert\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kx87v\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-kube-api-access-kx87v\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.358868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358774    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-podinfo\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.359172 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.358930    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-socket\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.359172 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.359035    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-credential-socket\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.359264 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.359152    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-certs\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.359334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.359316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-data\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.359465 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.359443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istiod-ca-cert\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.360856 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.360833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-envoy\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.361153 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.361133    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-podinfo\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.368260 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.368237    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-token\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.368817 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.368792    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kx87v\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-kube-api-access-kx87v\") pod \"e2e-api-b-057ec0-openshift-default-785446f97-sxjdr\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") " pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.414708 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.414678    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:46.558060 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.558037    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"]
Jul 21 18:38:46.560107 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:38:46.560080    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0c839a_8bf1_4370_ae34_0d811f3d39e8.slice/crio-503ec79a61537516396c1588a40eb109db51e95ff50938282e784fa89623cb18 WatchSource:0}: Error finding container 503ec79a61537516396c1588a40eb109db51e95ff50938282e784fa89623cb18: Status 404 returned error can't find the container with id 503ec79a61537516396c1588a40eb109db51e95ff50938282e784fa89623cb18
Jul 21 18:38:46.562342 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.562310    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:46.562426 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.562375    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:46.562426 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.562407    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:46.568447 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:46.568413    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" event={"ID":"6d0c839a-8bf1-4370-ae34-0d811f3d39e8","Type":"ContainerStarted","Data":"503ec79a61537516396c1588a40eb109db51e95ff50938282e784fa89623cb18"}
Jul 21 18:38:47.578669 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:47.578623    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" event={"ID":"6d0c839a-8bf1-4370-ae34-0d811f3d39e8","Type":"ContainerStarted","Data":"c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294"}
Jul 21 18:38:47.606038 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:47.605984    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" podStartSLOduration=1.6059674670000001 podStartE2EDuration="1.605967467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:38:47.60403414 +0000 UTC m=+2315.862477320" watchObservedRunningTime="2026-07-21 18:38:47.605967467 +0000 UTC m=+2315.864410646"
Jul 21 18:38:48.415705 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:48.415671    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:48.420649 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:48.420627    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:48.586636 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:48.586603    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:48.592053 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:48.592026    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:38:54.530939 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.530902    2578 patch_prober.go:28] interesting pod/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 21 18:38:54.531335 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.530957    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 21 18:38:54.617891 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.617856    2578 generic.go:356] "Generic (PLEG): container finished" podID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerID="d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46" exitCode=0
Jul 21 18:38:54.618051 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.617903    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" event={"ID":"01158ac3-e6ff-4171-b57d-9ec50024f740","Type":"ContainerDied","Data":"d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46"}
Jul 21 18:38:54.618357 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.618339    2578 scope.go:117] "RemoveContainer" containerID="d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46"
Jul 21 18:38:54.619412 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.619379    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:54.619518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.619449    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:54.619518 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:54.619496    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:38:55.624143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:55.624107    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" event={"ID":"01158ac3-e6ff-4171-b57d-9ec50024f740","Type":"ContainerStarted","Data":"aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7"}
Jul 21 18:38:56.408268 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:56.408233    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:38:56.408452 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:56.408401    2578 patch_prober.go:28] interesting pod/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 21 18:38:56.408497 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:56.408449    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 21 18:38:57.408246 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:57.408203    2578 patch_prober.go:28] interesting pod/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 21 18:38:57.408745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:57.408266    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 21 18:38:58.408317 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:58.408272    2578 patch_prober.go:28] interesting pod/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 21 18:38:58.408720 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:58.408330    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 21 18:38:59.407666 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:59.407624    2578 patch_prober.go:28] interesting pod/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Jul 21 18:38:59.407878 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:38:59.407691    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Jul 21 18:39:00.411759 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:00.411726    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:39:00.412122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:00.412032    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:39:00.412780 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:00.412759    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:39:04.628797 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:04.628752    2578 patch_prober.go:28] interesting pod/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 21 18:39:04.629209 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:04.628843    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 21 18:39:06.677155 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:06.677121    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerID="c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294" exitCode=0
Jul 21 18:39:06.677584 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:06.677193    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" event={"ID":"6d0c839a-8bf1-4370-ae34-0d811f3d39e8","Type":"ContainerDied","Data":"c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294"}
Jul 21 18:39:06.677584 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:06.677576    2578 scope.go:117] "RemoveContainer" containerID="c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294"
Jul 21 18:39:06.678638 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:06.678587    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:39:06.678758 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:06.678683    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:39:06.678758 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:06.678719    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:39:07.683693 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:07.683661    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" event={"ID":"6d0c839a-8bf1-4370-ae34-0d811f3d39e8","Type":"ContainerStarted","Data":"d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495"}
Jul 21 18:39:08.415033 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:08.414995    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:39:08.415268 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:08.415241    2578 patch_prober.go:28] interesting pod/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 21 18:39:08.415333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:08.415294    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 21 18:39:09.415895 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:09.415860    2578 patch_prober.go:28] interesting pod/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 21 18:39:09.416287 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:09.415914    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 21 18:39:10.415627 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:10.415571    2578 patch_prober.go:28] interesting pod/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 21 18:39:10.415814 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:10.415644    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 21 18:39:11.415984 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:11.415944    2578 patch_prober.go:28] interesting pod/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Jul 21 18:39:11.416387 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:11.416005    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Jul 21 18:39:12.419410 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:12.419380    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:39:12.419836 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:12.419760    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:39:12.420450 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:12.420433    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:39:29.962184 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:29.962154    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"]
Jul 21 18:39:29.962695 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:29.962381    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy" containerID="cri-o://aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7" gracePeriod=30
Jul 21 18:39:30.414467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:30.414425    2578 patch_prober.go:28] interesting pod/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 18:39:30.414662 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:30.414503    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 18:39:35.211401 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.211375    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:39:35.342562 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342482    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-podinfo\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342562 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342533    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vxd2\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-kube-api-access-5vxd2\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342562 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342563    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-credential-socket\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342647    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-socket\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342705    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-envoy\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342752    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-certs\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342781    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01158ac3-e6ff-4171-b57d-9ec50024f740-istiod-ca-cert\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342809    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-token\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.342848 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342842    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-data\") pod \"01158ac3-e6ff-4171-b57d-9ec50024f740\" (UID: \"01158ac3-e6ff-4171-b57d-9ec50024f740\") "
Jul 21 18:39:35.343168 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342959    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:39:35.343168 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.342977    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:39:35.343288 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.343171    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/01158ac3-e6ff-4171-b57d-9ec50024f740-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:39:35.343288 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.343218    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.343288 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.343238    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.343443 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.343398    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:39:35.343783 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.343761    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-data" (OuterVolumeSpecName: "istio-data") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:39:35.344844 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.344818    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-kube-api-access-5vxd2" (OuterVolumeSpecName: "kube-api-access-5vxd2") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "kube-api-access-5vxd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:39:35.344910 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.344895    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:39:35.345296 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.345270    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-token" (OuterVolumeSpecName: "istio-token") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:39:35.345339 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.345280    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "01158ac3-e6ff-4171-b57d-9ec50024f740" (UID: "01158ac3-e6ff-4171-b57d-9ec50024f740"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:39:35.443788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.443737    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.443788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.443782    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.443788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.443792    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/01158ac3-e6ff-4171-b57d-9ec50024f740-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.443788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.443800    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.443788 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.443808    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.444082 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.443815    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/01158ac3-e6ff-4171-b57d-9ec50024f740-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.444082 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.443823    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vxd2\" (UniqueName: \"kubernetes.io/projected/01158ac3-e6ff-4171-b57d-9ec50024f740-kube-api-access-5vxd2\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:39:35.825456 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.825424    2578 generic.go:356] "Generic (PLEG): container finished" podID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerID="aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7" exitCode=0
Jul 21 18:39:35.825728 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.825492    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"
Jul 21 18:39:35.825728 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.825517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" event={"ID":"01158ac3-e6ff-4171-b57d-9ec50024f740","Type":"ContainerDied","Data":"aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7"}
Jul 21 18:39:35.825728 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.825615    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6" event={"ID":"01158ac3-e6ff-4171-b57d-9ec50024f740","Type":"ContainerDied","Data":"300e1705a16ec6372bee65bd4ac8c10fd0842f6535c17a3855412c8b1fb676ab"}
Jul 21 18:39:35.825728 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.825641    2578 scope.go:117] "RemoveContainer" containerID="aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7"
Jul 21 18:39:35.834877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.834858    2578 scope.go:117] "RemoveContainer" containerID="d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46"
Jul 21 18:39:35.842569 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.842550    2578 scope.go:117] "RemoveContainer" containerID="aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7"
Jul 21 18:39:35.842837 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:39:35.842817    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7\": container with ID starting with aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7 not found: ID does not exist" containerID="aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7"
Jul 21 18:39:35.842920 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.842849    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7"} err="failed to get container status \"aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7\": rpc error: code = NotFound desc = could not find container \"aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7\": container with ID starting with aa2a2568a9384ce137e50cd4a4ff83a21f434eca526d85f0fe37a9e5700290c7 not found: ID does not exist"
Jul 21 18:39:35.842920 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.842874    2578 scope.go:117] "RemoveContainer" containerID="d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46"
Jul 21 18:39:35.843137 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:39:35.843119    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46\": container with ID starting with d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46 not found: ID does not exist" containerID="d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46"
Jul 21 18:39:35.843183 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.843143    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46"} err="failed to get container status \"d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46\": rpc error: code = NotFound desc = could not find container \"d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46\": container with ID starting with d7309a4f29c112a28a56f65606ad2eb5f3bd3f600a88f79888193775c2b14b46 not found: ID does not exist"
Jul 21 18:39:35.852686 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.852658    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"]
Jul 21 18:39:35.855447 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:35.855389    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-9c004f-openshift-default-59987455d7-bfww6"]
Jul 21 18:39:37.102011 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:37.101980    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" path="/var/lib/kubelet/pods/01158ac3-e6ff-4171-b57d-9ec50024f740/volumes"
Jul 21 18:39:58.172932 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:58.172898    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"]
Jul 21 18:39:58.173337 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:58.173111    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy" containerID="cri-o://d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495" gracePeriod=30
Jul 21 18:39:59.175818 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.175778    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"]
Jul 21 18:39:59.176236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176190    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy"
Jul 21 18:39:59.176236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176203    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca675797-babf-46a0-a767-f98191308516" containerName="istio-proxy"
Jul 21 18:39:59.176236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176225    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy"
Jul 21 18:39:59.176236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176232    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy"
Jul 21 18:39:59.176386 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176239    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy"
Jul 21 18:39:59.176386 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176246    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy"
Jul 21 18:39:59.176386 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176309    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy"
Jul 21 18:39:59.176386 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.176317    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="01158ac3-e6ff-4171-b57d-9ec50024f740" containerName="istio-proxy"
Jul 21 18:39:59.178326 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.178310    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.183353 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.183330    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c1a1f4-openshift-default-dockercfg-qjl2g\""
Jul 21 18:39:59.207451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.207420    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"]
Jul 21 18:39:59.261368 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261335    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-certs\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261536 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261404    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-envoy\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261536 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-socket\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261536 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261496    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-data\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261536 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261515    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-podinfo\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261536 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261531    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25p7k\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-kube-api-access-25p7k\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261737 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261552    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-credential-socket\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261737 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261671    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-token\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.261737 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.261702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575e52f8-c943-4c6e-a8b2-220e83afb292-istiod-ca-cert\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.362888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.362844    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-data\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.362888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.362891    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-podinfo\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363155 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.362991    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25p7k\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-kube-api-access-25p7k\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363155 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363011    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-credential-socket\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363265 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363212    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-token\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363265 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363253    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575e52f8-c943-4c6e-a8b2-220e83afb292-istiod-ca-cert\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363304    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-certs\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-data\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363345    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-credential-socket\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363408    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-envoy\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363494 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363485    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-socket\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363655 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363636    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-certs\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363853 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363833    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-socket\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.363934 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.363910    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575e52f8-c943-4c6e-a8b2-220e83afb292-istiod-ca-cert\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.369608 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.366001    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-podinfo\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.371867 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.371840    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-envoy\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.376305 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.376282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-token\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.377017 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.376993    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25p7k\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-kube-api-access-25p7k\") pod \"e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") " pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.489958 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.489872    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:39:59.639679 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.639646    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"]
Jul 21 18:39:59.643074 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:39:59.643048    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod575e52f8_c943_4c6e_a8b2_220e83afb292.slice/crio-95d72f97f29bc8a5e753cba711e86127fa95410603d2c49cd0c77db928b20be9 WatchSource:0}: Error finding container 95d72f97f29bc8a5e753cba711e86127fa95410603d2c49cd0c77db928b20be9: Status 404 returned error can't find the container with id 95d72f97f29bc8a5e753cba711e86127fa95410603d2c49cd0c77db928b20be9
Jul 21 18:39:59.645204 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.645163    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:39:59.645305 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.645235    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:39:59.645305 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.645261    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:39:59.955777 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.955737    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" event={"ID":"575e52f8-c943-4c6e-a8b2-220e83afb292","Type":"ContainerStarted","Data":"3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4"}
Jul 21 18:39:59.955980 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.955805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" event={"ID":"575e52f8-c943-4c6e-a8b2-220e83afb292","Type":"ContainerStarted","Data":"95d72f97f29bc8a5e753cba711e86127fa95410603d2c49cd0c77db928b20be9"}
Jul 21 18:39:59.982467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:39:59.982409    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" podStartSLOduration=0.982393144 podStartE2EDuration="982.393144ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:39:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:39:59.980053037 +0000 UTC m=+2388.238496218" watchObservedRunningTime="2026-07-21 18:39:59.982393144 +0000 UTC m=+2388.240836321"
Jul 21 18:40:00.491014 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:00.490977    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:00.495854 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:00.495829    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:00.960851 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:00.960814    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:00.961722 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:00.961702    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:03.426301 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.426278    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:40:03.506975 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.506892    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-data\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.506975 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.506924    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kx87v\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-kube-api-access-kx87v\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.506975 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.506945    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-envoy\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.507239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.506988    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-certs\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.507239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.507027    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-token\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.507239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.507065    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istiod-ca-cert\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.507239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.507093    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-credential-socket\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.507239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.507137    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-socket\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.507239 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.507164    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-podinfo\") pod \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\" (UID: \"6d0c839a-8bf1-4370-ae34-0d811f3d39e8\") "
Jul 21 18:40:03.508064 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.508018    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:40:03.508298 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.508236    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:40:03.508526 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.508407    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:40:03.508526 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.508465    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:40:03.514980 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.509635    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:40:03.514980 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.510253    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-kube-api-access-kx87v" (OuterVolumeSpecName: "kube-api-access-kx87v") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "kube-api-access-kx87v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:40:03.514980 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.514657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:40:03.517635 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.515716    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:40:03.517635 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.517114    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d0c839a-8bf1-4370-ae34-0d811f3d39e8" (UID: "6d0c839a-8bf1-4370-ae34-0d811f3d39e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:40:03.609907 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609870    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.609907 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609906    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kx87v\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-kube-api-access-kx87v\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.609907 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609918    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.609907 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609927    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.610179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609935    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.610179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609943    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.610179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609952    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.610179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609960    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.610179 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.609968    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d0c839a-8bf1-4370-ae34-0d811f3d39e8-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:40:03.975915 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.975877    2578 generic.go:356] "Generic (PLEG): container finished" podID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerID="d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495" exitCode=0
Jul 21 18:40:03.976128 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.975929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" event={"ID":"6d0c839a-8bf1-4370-ae34-0d811f3d39e8","Type":"ContainerDied","Data":"d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495"}
Jul 21 18:40:03.976128 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.975960    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"
Jul 21 18:40:03.976128 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.976026    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr" event={"ID":"6d0c839a-8bf1-4370-ae34-0d811f3d39e8","Type":"ContainerDied","Data":"503ec79a61537516396c1588a40eb109db51e95ff50938282e784fa89623cb18"}
Jul 21 18:40:03.976128 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.976056    2578 scope.go:117] "RemoveContainer" containerID="d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495"
Jul 21 18:40:03.987633 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.987613    2578 scope.go:117] "RemoveContainer" containerID="c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294"
Jul 21 18:40:03.996338 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.996320    2578 scope.go:117] "RemoveContainer" containerID="d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495"
Jul 21 18:40:03.996630 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:40:03.996606    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495\": container with ID starting with d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495 not found: ID does not exist" containerID="d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495"
Jul 21 18:40:03.996705 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.996642    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495"} err="failed to get container status \"d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495\": rpc error: code = NotFound desc = could not find container \"d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495\": container with ID starting with d218f861130450a72f0e30b56b6a60fe8e3074476b16b905de9e3e9f41fb4495 not found: ID does not exist"
Jul 21 18:40:03.996705 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.996661    2578 scope.go:117] "RemoveContainer" containerID="c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294"
Jul 21 18:40:03.996938 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:40:03.996917    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294\": container with ID starting with c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294 not found: ID does not exist" containerID="c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294"
Jul 21 18:40:03.996984 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:03.996946    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294"} err="failed to get container status \"c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294\": rpc error: code = NotFound desc = could not find container \"c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294\": container with ID starting with c2d0a6557c1f376b6c2cd7d15dcb67cdb911f33916b9e523023175f804614294 not found: ID does not exist"
Jul 21 18:40:04.001968 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:04.001943    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"]
Jul 21 18:40:04.006348 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:04.006324    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-057ec0-openshift-default-785446f97-sxjdr"]
Jul 21 18:40:05.101951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:05.101918    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" path="/var/lib/kubelet/pods/6d0c839a-8bf1-4370-ae34-0d811f3d39e8/volumes"
Jul 21 18:40:07.191226 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.191193    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"]
Jul 21 18:40:07.198878 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.198847    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy"
Jul 21 18:40:07.199041 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.199028    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy"
Jul 21 18:40:07.199162 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.199151    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy"
Jul 21 18:40:07.199250 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.199240    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy"
Jul 21 18:40:07.199521 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.199505    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy"
Jul 21 18:40:07.199668 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.199655    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0c839a-8bf1-4370-ae34-0d811f3d39e8" containerName="istio-proxy"
Jul 21 18:40:07.203032 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.203010    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"]
Jul 21 18:40:07.203308 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.203246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.209032 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.208939    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-0be774-openshift-default-dockercfg-nlqt4\""
Jul 21 18:40:07.340789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.340749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-token\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.340959 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.340803    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4tsr\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-kube-api-access-f4tsr\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.340959 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.340912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a920794-bae8-4698-b6b2-1ae315ca3b80-istiod-ca-cert\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.340959 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.340942    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-data\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.341070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.341004    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-socket\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.341070 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.341043    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-credential-socket\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.341134 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.341068    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-envoy\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.341134 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.341090    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-certs\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.341201 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.341171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-podinfo\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442289 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442189    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-token\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442289 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4tsr\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-kube-api-access-f4tsr\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442289 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442285    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a920794-bae8-4698-b6b2-1ae315ca3b80-istiod-ca-cert\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-data\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442359    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-socket\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442390    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-credential-socket\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442412    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-envoy\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442438    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-certs\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442650 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442516    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-podinfo\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442997 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442879    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-socket\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442997 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442939    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-data\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.442997 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.442980    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-credential-socket\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.443149 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.443012    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-certs\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.443149 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.443098    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a920794-bae8-4698-b6b2-1ae315ca3b80-istiod-ca-cert\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.444873 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.444845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-envoy\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.445245 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.445218    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-podinfo\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.455307 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.455282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-token\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.455439 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.455418    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4tsr\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-kube-api-access-f4tsr\") pod \"e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") " pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.519622 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.519552    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:07.686464 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.686346    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"]
Jul 21 18:40:07.695803 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:40:07.695773    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a920794_bae8_4698_b6b2_1ae315ca3b80.slice/crio-860da994b09329496b90a98ec3cd1a6b6e83dec0065afcb7b9c40fbcdd21db3c WatchSource:0}: Error finding container 860da994b09329496b90a98ec3cd1a6b6e83dec0065afcb7b9c40fbcdd21db3c: Status 404 returned error can't find the container with id 860da994b09329496b90a98ec3cd1a6b6e83dec0065afcb7b9c40fbcdd21db3c
Jul 21 18:40:07.698132 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.698091    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:07.698225 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.698186    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:07.698262 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:07.698245    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:08.001956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:08.001833    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" event={"ID":"1a920794-bae8-4698-b6b2-1ae315ca3b80","Type":"ContainerStarted","Data":"19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9"}
Jul 21 18:40:08.001956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:08.001891    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" event={"ID":"1a920794-bae8-4698-b6b2-1ae315ca3b80","Type":"ContainerStarted","Data":"860da994b09329496b90a98ec3cd1a6b6e83dec0065afcb7b9c40fbcdd21db3c"}
Jul 21 18:40:08.031321 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:08.031266    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" podStartSLOduration=1.031247679 podStartE2EDuration="1.031247679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:40:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:40:08.028338925 +0000 UTC m=+2396.286782105" watchObservedRunningTime="2026-07-21 18:40:08.031247679 +0000 UTC m=+2396.289690858"
Jul 21 18:40:08.520414 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:08.520384    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:08.525382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:08.525359    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:09.006939 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:09.006910    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:09.007971 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:09.007952    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:13.027763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.027690    2578 generic.go:356] "Generic (PLEG): container finished" podID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerID="3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4" exitCode=0
Jul 21 18:40:13.028174 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.027759    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" event={"ID":"575e52f8-c943-4c6e-a8b2-220e83afb292","Type":"ContainerDied","Data":"3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4"}
Jul 21 18:40:13.028220 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.028185    2578 scope.go:117] "RemoveContainer" containerID="3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4"
Jul 21 18:40:13.029231 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.029194    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:13.029306 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.029284    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:13.029346 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.029332    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:13.616339 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.616306    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:40:13.644004 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:13.643314    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:40:14.035529 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:14.035490    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" event={"ID":"575e52f8-c943-4c6e-a8b2-220e83afb292","Type":"ContainerStarted","Data":"c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e"}
Jul 21 18:40:14.490530 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:14.490487    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:14.491867 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:14.491831    2578 patch_prober.go:28] interesting pod/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Jul 21 18:40:14.492018 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:14.491901    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Jul 21 18:40:15.490535 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:15.490488    2578 patch_prober.go:28] interesting pod/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Jul 21 18:40:15.491055 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:15.490557    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Jul 21 18:40:15.960952 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:15.960912    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:16.491193 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:16.491155    2578 patch_prober.go:28] interesting pod/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Jul 21 18:40:16.491820 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:16.491785    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Jul 21 18:40:17.491220 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:17.491181    2578 patch_prober.go:28] interesting pod/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body=
Jul 21 18:40:17.491837 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:17.491805    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused"
Jul 21 18:40:18.494195 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:18.494166    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:18.495336 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:18.495312    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:40:21.069645 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:21.069609    2578 generic.go:356] "Generic (PLEG): container finished" podID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerID="19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9" exitCode=0
Jul 21 18:40:21.070118 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:21.069689    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" event={"ID":"1a920794-bae8-4698-b6b2-1ae315ca3b80","Type":"ContainerDied","Data":"19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9"}
Jul 21 18:40:21.070231 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:21.070212    2578 scope.go:117] "RemoveContainer" containerID="19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9"
Jul 21 18:40:21.071319 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:21.071291    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:21.071389 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:21.071363    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:21.071425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:21.071395    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 21 18:40:22.082872 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:22.082837    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" event={"ID":"1a920794-bae8-4698-b6b2-1ae315ca3b80","Type":"ContainerStarted","Data":"3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556"}
Jul 21 18:40:22.520691 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:22.520645    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:22.521853 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:22.521826    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 21 18:40:22.521989 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:22.521877    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 21 18:40:23.520232 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:23.520196    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 21 18:40:23.520703 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:23.520263    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 21 18:40:24.007807 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:24.007759    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:24.520262 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:24.520224    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 21 18:40:24.520689 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:24.520281    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 21 18:40:25.520984 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:25.520943    2578 patch_prober.go:28] interesting pod/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Jul 21 18:40:25.521376 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:25.521004    2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Jul 21 18:40:26.524255 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:26.524227    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:26.525099 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:26.525078    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:40:34.840490 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.840413    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"]
Jul 21 18:40:34.842890 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.842873    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:34.845823 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.845801    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"openshift-service-ca.crt\""
Jul 21 18:40:34.845955 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.845842    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"kube-root-ca.crt\""
Jul 21 18:40:34.847359 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.847336    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"auth-test-model-c1a1f4-kserve-self-signed-certs\""
Jul 21 18:40:34.847470 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.847363    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-c1a1f4\"/\"default-dockercfg-j2vk6\""
Jul 21 18:40:34.854637 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.854611    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"]
Jul 21 18:40:34.904176 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.904147    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl5qk\" (UniqueName: \"kubernetes.io/projected/3eac010d-cd8b-41cc-9123-e99865452ba8-kube-api-access-hl5qk\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:34.904176 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.904181    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-dshm\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:34.904402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.904215    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-model-cache\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:34.904402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.904270    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-home\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:34.904402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.904329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-tmp-dir\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:34.904402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:34.904349    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eac010d-cd8b-41cc-9123-e99865452ba8-tls-certs\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.005698 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.005661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-model-cache\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.005698 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.005699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-home\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.005951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.005730    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-tmp-dir\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.005951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.005746    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eac010d-cd8b-41cc-9123-e99865452ba8-tls-certs\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.005951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.005798    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl5qk\" (UniqueName: \"kubernetes.io/projected/3eac010d-cd8b-41cc-9123-e99865452ba8-kube-api-access-hl5qk\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.005951 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.005817    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-dshm\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.006220 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.006186    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-model-cache\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.006298 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.006237    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-home\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.006298 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.006268    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-tmp-dir\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.008128 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.008108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-dshm\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.008316 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.008299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eac010d-cd8b-41cc-9123-e99865452ba8-tls-certs\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.014994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.014969    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl5qk\" (UniqueName: \"kubernetes.io/projected/3eac010d-cd8b-41cc-9123-e99865452ba8-kube-api-access-hl5qk\") pod \"auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") " pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.152868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.152839    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:35.284241 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:35.284218    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"]
Jul 21 18:40:35.286307 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:40:35.286280    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3eac010d_cd8b_41cc_9123_e99865452ba8.slice/crio-21a5b24f86c15258c7cb239132c750fa86b6423c7d2f351baa41283e69650bfd WatchSource:0}: Error finding container 21a5b24f86c15258c7cb239132c750fa86b6423c7d2f351baa41283e69650bfd: Status 404 returned error can't find the container with id 21a5b24f86c15258c7cb239132c750fa86b6423c7d2f351baa41283e69650bfd
Jul 21 18:40:36.152883 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:36.152848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7" event={"ID":"3eac010d-cd8b-41cc-9123-e99865452ba8","Type":"ContainerStarted","Data":"4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162"}
Jul 21 18:40:36.152883 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:36.152888    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7" event={"ID":"3eac010d-cd8b-41cc-9123-e99865452ba8","Type":"ContainerStarted","Data":"21a5b24f86c15258c7cb239132c750fa86b6423c7d2f351baa41283e69650bfd"}
Jul 21 18:40:36.173432 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:36.173386    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7" podStartSLOduration=2.1733717439999998 podStartE2EDuration="2.173371744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:40:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:40:36.171712431 +0000 UTC m=+2424.430155614" watchObservedRunningTime="2026-07-21 18:40:36.173371744 +0000 UTC m=+2424.431814925"
Jul 21 18:40:45.153713 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:45.153658    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:45.154243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:45.153729    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:45.166796 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:45.166765    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:40:45.209330 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:40:45.209303    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:42:56.086559 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:42:56.086519    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"]
Jul 21 18:43:11.106563 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.106523    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7" podUID="3eac010d-cd8b-41cc-9123-e99865452ba8" containerName="main" containerID="cri-o://4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162" gracePeriod=45
Jul 21 18:43:11.362463 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.362406    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:43:11.481690 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481656    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-tmp-dir\") pod \"3eac010d-cd8b-41cc-9123-e99865452ba8\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") "
Jul 21 18:43:11.481867 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481751    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-home\") pod \"3eac010d-cd8b-41cc-9123-e99865452ba8\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") "
Jul 21 18:43:11.481867 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481782    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl5qk\" (UniqueName: \"kubernetes.io/projected/3eac010d-cd8b-41cc-9123-e99865452ba8-kube-api-access-hl5qk\") pod \"3eac010d-cd8b-41cc-9123-e99865452ba8\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") "
Jul 21 18:43:11.481867 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-dshm\") pod \"3eac010d-cd8b-41cc-9123-e99865452ba8\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") "
Jul 21 18:43:11.481867 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481820    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-model-cache\") pod \"3eac010d-cd8b-41cc-9123-e99865452ba8\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") "
Jul 21 18:43:11.481867 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481839    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eac010d-cd8b-41cc-9123-e99865452ba8-tls-certs\") pod \"3eac010d-cd8b-41cc-9123-e99865452ba8\" (UID: \"3eac010d-cd8b-41cc-9123-e99865452ba8\") "
Jul 21 18:43:11.482133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481950    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3eac010d-cd8b-41cc-9123-e99865452ba8" (UID: "3eac010d-cd8b-41cc-9123-e99865452ba8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:43:11.482133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.481995    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-home" (OuterVolumeSpecName: "home") pod "3eac010d-cd8b-41cc-9123-e99865452ba8" (UID: "3eac010d-cd8b-41cc-9123-e99865452ba8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:43:11.482133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.482104    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-home\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:43:11.482133 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.482124    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-tmp-dir\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:43:11.482285 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.482135    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-model-cache" (OuterVolumeSpecName: "model-cache") pod "3eac010d-cd8b-41cc-9123-e99865452ba8" (UID: "3eac010d-cd8b-41cc-9123-e99865452ba8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:43:11.483987 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.483964    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-dshm" (OuterVolumeSpecName: "dshm") pod "3eac010d-cd8b-41cc-9123-e99865452ba8" (UID: "3eac010d-cd8b-41cc-9123-e99865452ba8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:43:11.484111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.484015    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3eac010d-cd8b-41cc-9123-e99865452ba8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3eac010d-cd8b-41cc-9123-e99865452ba8" (UID: "3eac010d-cd8b-41cc-9123-e99865452ba8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:43:11.484111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.484070    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3eac010d-cd8b-41cc-9123-e99865452ba8-kube-api-access-hl5qk" (OuterVolumeSpecName: "kube-api-access-hl5qk") pod "3eac010d-cd8b-41cc-9123-e99865452ba8" (UID: "3eac010d-cd8b-41cc-9123-e99865452ba8"). InnerVolumeSpecName "kube-api-access-hl5qk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:43:11.583090 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.583051    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl5qk\" (UniqueName: \"kubernetes.io/projected/3eac010d-cd8b-41cc-9123-e99865452ba8-kube-api-access-hl5qk\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:43:11.583090 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.583084    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-dshm\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:43:11.583090 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.583096    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3eac010d-cd8b-41cc-9123-e99865452ba8-model-cache\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:43:11.583309 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.583108    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3eac010d-cd8b-41cc-9123-e99865452ba8-tls-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:43:11.900651 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.900612    2578 generic.go:356] "Generic (PLEG): container finished" podID="3eac010d-cd8b-41cc-9123-e99865452ba8" containerID="4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162" exitCode=0
Jul 21 18:43:11.900812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.900672    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"
Jul 21 18:43:11.900812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.900672    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7" event={"ID":"3eac010d-cd8b-41cc-9123-e99865452ba8","Type":"ContainerDied","Data":"4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162"}
Jul 21 18:43:11.900812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.900792    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7" event={"ID":"3eac010d-cd8b-41cc-9123-e99865452ba8","Type":"ContainerDied","Data":"21a5b24f86c15258c7cb239132c750fa86b6423c7d2f351baa41283e69650bfd"}
Jul 21 18:43:11.900933 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.900819    2578 scope.go:117] "RemoveContainer" containerID="4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162"
Jul 21 18:43:11.909747 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.909727    2578 scope.go:117] "RemoveContainer" containerID="4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162"
Jul 21 18:43:11.910008 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:43:11.909990    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162\": container with ID starting with 4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162 not found: ID does not exist" containerID="4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162"
Jul 21 18:43:11.910061 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.910016    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162"} err="failed to get container status \"4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162\": rpc error: code = NotFound desc = could not find container \"4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162\": container with ID starting with 4b673ea955f00aed59579f0f9a25478f8bc60ff300403e21e466805460720162 not found: ID does not exist"
Jul 21 18:43:11.922561 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.922538    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"]
Jul 21 18:43:11.926432 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:11.926413    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-c1a1f4/auth-test-model-c1a1f4-kserve-654f7fccf5-cjtb7"]
Jul 21 18:43:13.102640 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:13.102605    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3eac010d-cd8b-41cc-9123-e99865452ba8" path="/var/lib/kubelet/pods/3eac010d-cd8b-41cc-9123-e99865452ba8/volumes"
Jul 21 18:43:40.533877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.533788    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"]
Jul 21 18:43:40.534467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.534372    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3eac010d-cd8b-41cc-9123-e99865452ba8" containerName="main"
Jul 21 18:43:40.534467 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.534390    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3eac010d-cd8b-41cc-9123-e99865452ba8" containerName="main"
Jul 21 18:43:40.534717 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.534503    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3eac010d-cd8b-41cc-9123-e99865452ba8" containerName="main"
Jul 21 18:43:40.537559 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.537536    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.540406 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.540386    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"openshift-service-ca.crt\""
Jul 21 18:43:40.541853 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.541834    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"kube-root-ca.crt\""
Jul 21 18:43:40.542160 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.542064    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"sub-test-model-0be774-kserve-self-signed-certs\""
Jul 21 18:43:40.543038 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.543012    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"default-dockercfg-fcm8s\""
Jul 21 18:43:40.544422 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.544401    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"]
Jul 21 18:43:40.651059 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.651019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-home\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.651243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.651067    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tmp-dir\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.651243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.651098    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tls-certs\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.651243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.651116    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdpcf\" (UniqueName: \"kubernetes.io/projected/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-kube-api-access-qdpcf\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.651243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.651138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-dshm\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.651243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.651204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-model-cache\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.751889 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.751857    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-model-cache\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752073 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.751910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-home\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752073 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.751937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tmp-dir\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752073 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.751967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tls-certs\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752073 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.751985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdpcf\" (UniqueName: \"kubernetes.io/projected/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-kube-api-access-qdpcf\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752073 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.752008    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-dshm\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752342 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.752316    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-model-cache\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752396 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.752352    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-home\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.752437 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.752391    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tmp-dir\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.754279 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.754254    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-dshm\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.754492 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.754472    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tls-certs\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.761178 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.761151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdpcf\" (UniqueName: \"kubernetes.io/projected/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-kube-api-access-qdpcf\") pod \"sub-test-model-0be774-kserve-84498567d5-qgx45\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") " pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.850134 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.850031    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:40.986228 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:40.986192    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"]
Jul 21 18:43:40.988236 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:43:40.988205    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode10f8aa6_1e17_46dc_a42f_b093cac5bbd4.slice/crio-fc3cb014a27dcc60b0660adc567a7227a6c9a10c3f12ec9d82dac105aec1c724 WatchSource:0}: Error finding container fc3cb014a27dcc60b0660adc567a7227a6c9a10c3f12ec9d82dac105aec1c724: Status 404 returned error can't find the container with id fc3cb014a27dcc60b0660adc567a7227a6c9a10c3f12ec9d82dac105aec1c724
Jul 21 18:43:41.040782 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:41.040753    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45" event={"ID":"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4","Type":"ContainerStarted","Data":"fc3cb014a27dcc60b0660adc567a7227a6c9a10c3f12ec9d82dac105aec1c724"}
Jul 21 18:43:42.047340 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:42.047307    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45" event={"ID":"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4","Type":"ContainerStarted","Data":"6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379"}
Jul 21 18:43:42.066373 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:42.066312    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45" podStartSLOduration=2.066294361 podStartE2EDuration="2.066294361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:43:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:43:42.064173309 +0000 UTC m=+2610.322616491" watchObservedRunningTime="2026-07-21 18:43:42.066294361 +0000 UTC m=+2610.324737541"
Jul 21 18:43:50.851007 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:50.850970    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:50.851474 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:50.851019    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:50.863576 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:50.863544    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:43:51.102371 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:43:51.102294    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:44:30.472119 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:30.472082    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"]
Jul 21 18:44:45.492236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.492198    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45" podUID="e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" containerName="main" containerID="cri-o://6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379" gracePeriod=45
Jul 21 18:44:45.745102 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.745039    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:44:45.850026 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.849993    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tls-certs\") pod \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") "
Jul 21 18:44:45.850192 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850067    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-model-cache\") pod \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") "
Jul 21 18:44:45.850192 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850117    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-home\") pod \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") "
Jul 21 18:44:45.850192 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850143    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-dshm\") pod \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") "
Jul 21 18:44:45.850192 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850182    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tmp-dir\") pod \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") "
Jul 21 18:44:45.850358 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850211    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdpcf\" (UniqueName: \"kubernetes.io/projected/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-kube-api-access-qdpcf\") pod \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\" (UID: \"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4\") "
Jul 21 18:44:45.850414 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850380    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-model-cache" (OuterVolumeSpecName: "model-cache") pod "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" (UID: "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:44:45.850472 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850386    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-home" (OuterVolumeSpecName: "home") pod "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" (UID: "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:44:45.850472 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850449    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" (UID: "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:44:45.850583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850535    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-home\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:44:45.850583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850550    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tmp-dir\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:44:45.850583 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.850559    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-model-cache\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:44:45.852284 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.852255    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-dshm" (OuterVolumeSpecName: "dshm") pod "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" (UID: "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:44:45.852398 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.852335    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-kube-api-access-qdpcf" (OuterVolumeSpecName: "kube-api-access-qdpcf") pod "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" (UID: "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4"). InnerVolumeSpecName "kube-api-access-qdpcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:44:45.852446 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.852417    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" (UID: "e10f8aa6-1e17-46dc-a42f-b093cac5bbd4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:44:45.950950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.950911    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-tls-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:44:45.950950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.950936    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-dshm\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:44:45.950950 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:45.950949    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdpcf\" (UniqueName: \"kubernetes.io/projected/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4-kube-api-access-qdpcf\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:44:46.356309 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.356274    2578 generic.go:356] "Generic (PLEG): container finished" podID="e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" containerID="6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379" exitCode=0
Jul 21 18:44:46.356503 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.356333    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"
Jul 21 18:44:46.356503 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.356378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45" event={"ID":"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4","Type":"ContainerDied","Data":"6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379"}
Jul 21 18:44:46.356503 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.356424    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45" event={"ID":"e10f8aa6-1e17-46dc-a42f-b093cac5bbd4","Type":"ContainerDied","Data":"fc3cb014a27dcc60b0660adc567a7227a6c9a10c3f12ec9d82dac105aec1c724"}
Jul 21 18:44:46.356503 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.356445    2578 scope.go:117] "RemoveContainer" containerID="6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379"
Jul 21 18:44:46.365582 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.365563    2578 scope.go:117] "RemoveContainer" containerID="6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379"
Jul 21 18:44:46.365882 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:44:46.365861    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379\": container with ID starting with 6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379 not found: ID does not exist" containerID="6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379"
Jul 21 18:44:46.365994 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.365887    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379"} err="failed to get container status \"6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379\": rpc error: code = NotFound desc = could not find container \"6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379\": container with ID starting with 6c4031627ec5dd6cb37013ac76df6a6d64901c3b4e68a08a60f885b5c10bc379 not found: ID does not exist"
Jul 21 18:44:46.377711 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.377690    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"]
Jul 21 18:44:46.383870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:46.383847    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/sub-test-model-0be774-kserve-84498567d5-qgx45"]
Jul 21 18:44:47.102333 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:44:47.102292    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" path="/var/lib/kubelet/pods/e10f8aa6-1e17-46dc-a42f-b093cac5bbd4/volumes"
Jul 21 18:45:12.529779 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.529654    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"]
Jul 21 18:45:12.530326 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.530163    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" containerName="main"
Jul 21 18:45:12.530326 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.530185    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" containerName="main"
Jul 21 18:45:12.530326 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.530283    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e10f8aa6-1e17-46dc-a42f-b093cac5bbd4" containerName="main"
Jul 21 18:45:12.533114 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.533098    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.537236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.537216    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"default-dockercfg-fcm8s\""
Jul 21 18:45:12.537236 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.537230    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"kube-root-ca.crt\""
Jul 21 18:45:12.537478 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.537270    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"openshift-service-ca.crt\""
Jul 21 18:45:12.537478 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.537345    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-0be774\"/\"rate-test-model-0be774-kserve-self-signed-certs\""
Jul 21 18:45:12.543928 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.543906    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"]
Jul 21 18:45:12.598764 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.598727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-model-cache\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.598937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.598773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-home\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.598937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.598825    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-tmp-dir\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.598937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.598901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-dshm\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.598937 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.598930    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df07351f-32a8-4191-8d82-3c5f68295726-tls-certs\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.599096 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.598950    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrgn2\" (UniqueName: \"kubernetes.io/projected/df07351f-32a8-4191-8d82-3c5f68295726-kube-api-access-vrgn2\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.699465 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.699433    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-tmp-dir\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.699643 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.699512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-dshm\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.699643 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.699539    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df07351f-32a8-4191-8d82-3c5f68295726-tls-certs\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.699643 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.699557    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrgn2\" (UniqueName: \"kubernetes.io/projected/df07351f-32a8-4191-8d82-3c5f68295726-kube-api-access-vrgn2\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.699810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.699738    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-model-cache\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.699870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.699834    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-home\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.699870 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.699832    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-tmp-dir\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.700095 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.700073    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-model-cache\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.700213 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.700191    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-home\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.701792 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.701769    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-dshm\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.701954 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.701934    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df07351f-32a8-4191-8d82-3c5f68295726-tls-certs\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.708363 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.708338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrgn2\" (UniqueName: \"kubernetes.io/projected/df07351f-32a8-4191-8d82-3c5f68295726-kube-api-access-vrgn2\") pod \"rate-test-model-0be774-kserve-6c59587bf5-h2gzv\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") " pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.842506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.842402    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:12.991633 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:12.990629    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"]
Jul 21 18:45:13.491525 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:13.491473    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv" event={"ID":"df07351f-32a8-4191-8d82-3c5f68295726","Type":"ContainerStarted","Data":"1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4"}
Jul 21 18:45:13.491763 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:13.491536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv" event={"ID":"df07351f-32a8-4191-8d82-3c5f68295726","Type":"ContainerStarted","Data":"a622b5e5b54c2e551d97ca3d139f85a41363910a3ebf1eb2b75f727e91225706"}
Jul 21 18:45:13.511370 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:13.511313    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv" podStartSLOduration=1.511297988 podStartE2EDuration="1.511297988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:45:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:45:13.509157656 +0000 UTC m=+2701.767600827" watchObservedRunningTime="2026-07-21 18:45:13.511297988 +0000 UTC m=+2701.769741166"
Jul 21 18:45:13.663786 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:13.663758    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:45:13.677271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:13.677243    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7dgs2_f3be15ef-cbc8-43ee-862b-c8b341d07fb4/ovn-acl-logging/0.log"
Jul 21 18:45:22.843654 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:22.843615    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:22.844088 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:22.843670    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:22.856151 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:22.856128    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:23.549812 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:23.549786    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:45:56.139885 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:45:56.139851    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"]
Jul 21 18:46:11.160449 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.160401    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv" podUID="df07351f-32a8-4191-8d82-3c5f68295726" containerName="main" containerID="cri-o://1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4" gracePeriod=45
Jul 21 18:46:11.423111 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.423050    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:46:11.567231 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567201    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-model-cache\") pod \"df07351f-32a8-4191-8d82-3c5f68295726\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") "
Jul 21 18:46:11.567421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567275    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-tmp-dir\") pod \"df07351f-32a8-4191-8d82-3c5f68295726\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") "
Jul 21 18:46:11.567421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567305    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrgn2\" (UniqueName: \"kubernetes.io/projected/df07351f-32a8-4191-8d82-3c5f68295726-kube-api-access-vrgn2\") pod \"df07351f-32a8-4191-8d82-3c5f68295726\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") "
Jul 21 18:46:11.567421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567352    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-dshm\") pod \"df07351f-32a8-4191-8d82-3c5f68295726\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") "
Jul 21 18:46:11.567421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567385    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df07351f-32a8-4191-8d82-3c5f68295726-tls-certs\") pod \"df07351f-32a8-4191-8d82-3c5f68295726\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") "
Jul 21 18:46:11.567421 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567417    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-home\") pod \"df07351f-32a8-4191-8d82-3c5f68295726\" (UID: \"df07351f-32a8-4191-8d82-3c5f68295726\") "
Jul 21 18:46:11.567724 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567509    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-model-cache" (OuterVolumeSpecName: "model-cache") pod "df07351f-32a8-4191-8d82-3c5f68295726" (UID: "df07351f-32a8-4191-8d82-3c5f68295726"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:11.567724 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567562    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df07351f-32a8-4191-8d82-3c5f68295726" (UID: "df07351f-32a8-4191-8d82-3c5f68295726"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:11.567839 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567775    2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-tmp-dir\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:11.567839 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567795    2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-model-cache\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:11.567839 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.567786    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-home" (OuterVolumeSpecName: "home") pod "df07351f-32a8-4191-8d82-3c5f68295726" (UID: "df07351f-32a8-4191-8d82-3c5f68295726"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:11.569424 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.569400    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-dshm" (OuterVolumeSpecName: "dshm") pod "df07351f-32a8-4191-8d82-3c5f68295726" (UID: "df07351f-32a8-4191-8d82-3c5f68295726"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:11.569510 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.569469    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df07351f-32a8-4191-8d82-3c5f68295726-kube-api-access-vrgn2" (OuterVolumeSpecName: "kube-api-access-vrgn2") pod "df07351f-32a8-4191-8d82-3c5f68295726" (UID: "df07351f-32a8-4191-8d82-3c5f68295726"). InnerVolumeSpecName "kube-api-access-vrgn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:46:11.569555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.569505    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df07351f-32a8-4191-8d82-3c5f68295726-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df07351f-32a8-4191-8d82-3c5f68295726" (UID: "df07351f-32a8-4191-8d82-3c5f68295726"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 18:46:11.669096 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.669056    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrgn2\" (UniqueName: \"kubernetes.io/projected/df07351f-32a8-4191-8d82-3c5f68295726-kube-api-access-vrgn2\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:11.669096 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.669091    2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-dshm\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:11.669096 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.669102    2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df07351f-32a8-4191-8d82-3c5f68295726-tls-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:11.669358 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.669113    2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df07351f-32a8-4191-8d82-3c5f68295726-home\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:11.768334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.768245    2578 generic.go:356] "Generic (PLEG): container finished" podID="df07351f-32a8-4191-8d82-3c5f68295726" containerID="1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4" exitCode=0
Jul 21 18:46:11.768334 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.768298    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"
Jul 21 18:46:11.768506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.768328    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv" event={"ID":"df07351f-32a8-4191-8d82-3c5f68295726","Type":"ContainerDied","Data":"1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4"}
Jul 21 18:46:11.768506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.768375    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv" event={"ID":"df07351f-32a8-4191-8d82-3c5f68295726","Type":"ContainerDied","Data":"a622b5e5b54c2e551d97ca3d139f85a41363910a3ebf1eb2b75f727e91225706"}
Jul 21 18:46:11.768506 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.768392    2578 scope.go:117] "RemoveContainer" containerID="1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4"
Jul 21 18:46:11.777471 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.777454    2578 scope.go:117] "RemoveContainer" containerID="1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4"
Jul 21 18:46:11.777728 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:46:11.777708    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4\": container with ID starting with 1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4 not found: ID does not exist" containerID="1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4"
Jul 21 18:46:11.777806 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.777735    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4"} err="failed to get container status \"1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4\": rpc error: code = NotFound desc = could not find container \"1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4\": container with ID starting with 1fa71095a65477a3adb57eaf225f10bdd6b82490ecca5ca223510594ba37e4f4 not found: ID does not exist"
Jul 21 18:46:11.789791 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.789769    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"]
Jul 21 18:46:11.791631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:11.791612    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-0be774/rate-test-model-0be774-kserve-6c59587bf5-h2gzv"]
Jul 21 18:46:12.052728 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.052644    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7562-140036458792976"]
Jul 21 18:46:12.053026 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.053013    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df07351f-32a8-4191-8d82-3c5f68295726" containerName="main"
Jul 21 18:46:12.053069 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.053027    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="df07351f-32a8-4191-8d82-3c5f68295726" containerName="main"
Jul 21 18:46:12.053104 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.053090    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="df07351f-32a8-4191-8d82-3c5f68295726" containerName="main"
Jul 21 18:46:12.055888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.055870    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976"
Jul 21 18:46:12.058540 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.058516    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-lfsn8\""
Jul 21 18:46:12.060141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.060119    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"]
Jul 21 18:46:12.174035 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.174002    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j62jf\" (UniqueName: \"kubernetes.io/projected/c191734c-168e-4d2d-8127-0a824d243685-kube-api-access-j62jf\") pod \"test-curl-7562-140036458792976\" (UID: \"c191734c-168e-4d2d-8127-0a824d243685\") " pod="opendatahub/test-curl-7562-140036458792976"
Jul 21 18:46:12.274653 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.274616    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j62jf\" (UniqueName: \"kubernetes.io/projected/c191734c-168e-4d2d-8127-0a824d243685-kube-api-access-j62jf\") pod \"test-curl-7562-140036458792976\" (UID: \"c191734c-168e-4d2d-8127-0a824d243685\") " pod="opendatahub/test-curl-7562-140036458792976"
Jul 21 18:46:12.283680 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.283659    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j62jf\" (UniqueName: \"kubernetes.io/projected/c191734c-168e-4d2d-8127-0a824d243685-kube-api-access-j62jf\") pod \"test-curl-7562-140036458792976\" (UID: \"c191734c-168e-4d2d-8127-0a824d243685\") " pod="opendatahub/test-curl-7562-140036458792976"
Jul 21 18:46:12.367277 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.367198    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976"
Jul 21 18:46:12.502578 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.502550    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"]
Jul 21 18:46:12.507283 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:46:12.507240    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc191734c_168e_4d2d_8127_0a824d243685.slice/crio-0a09a21262f92c223f9c9d97ce40aaf7f4c60e6d640680e00fc99142ffa982cb WatchSource:0}: Error finding container 0a09a21262f92c223f9c9d97ce40aaf7f4c60e6d640680e00fc99142ffa982cb: Status 404 returned error can't find the container with id 0a09a21262f92c223f9c9d97ce40aaf7f4c60e6d640680e00fc99142ffa982cb
Jul 21 18:46:12.510429 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.510395    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 18:46:12.776045 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:12.776009    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458792976" event={"ID":"c191734c-168e-4d2d-8127-0a824d243685","Type":"ContainerStarted","Data":"0a09a21262f92c223f9c9d97ce40aaf7f4c60e6d640680e00fc99142ffa982cb"}
Jul 21 18:46:13.104382 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:13.104289    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df07351f-32a8-4191-8d82-3c5f68295726" path="/var/lib/kubelet/pods/df07351f-32a8-4191-8d82-3c5f68295726/volumes"
Jul 21 18:46:13.781455 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:13.781414    2578 generic.go:356] "Generic (PLEG): container finished" podID="c191734c-168e-4d2d-8127-0a824d243685" containerID="e6aabbfce17e515493769d4c2750f670540471f208dc882f9583150104e50543" exitCode=0
Jul 21 18:46:13.781868 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:13.781502    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7562-140036458792976" event={"ID":"c191734c-168e-4d2d-8127-0a824d243685","Type":"ContainerDied","Data":"e6aabbfce17e515493769d4c2750f670540471f208dc882f9583150104e50543"}
Jul 21 18:46:14.921966 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:14.921939    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976"
Jul 21 18:46:14.957908 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:14.957873    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7562-140036458792976_c191734c-168e-4d2d-8127-0a824d243685/test-curl-7562-140036458792976/0.log"
Jul 21 18:46:14.984286 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:14.984252    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"]
Jul 21 18:46:14.986017 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:14.985992    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7562-140036458792976"]
Jul 21 18:46:14.996829 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:14.996806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j62jf\" (UniqueName: \"kubernetes.io/projected/c191734c-168e-4d2d-8127-0a824d243685-kube-api-access-j62jf\") pod \"c191734c-168e-4d2d-8127-0a824d243685\" (UID: \"c191734c-168e-4d2d-8127-0a824d243685\") "
Jul 21 18:46:14.998833 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:14.998813    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c191734c-168e-4d2d-8127-0a824d243685-kube-api-access-j62jf" (OuterVolumeSpecName: "kube-api-access-j62jf") pod "c191734c-168e-4d2d-8127-0a824d243685" (UID: "c191734c-168e-4d2d-8127-0a824d243685"). InnerVolumeSpecName "kube-api-access-j62jf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:46:15.097652 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:15.097573    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j62jf\" (UniqueName: \"kubernetes.io/projected/c191734c-168e-4d2d-8127-0a824d243685-kube-api-access-j62jf\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:15.102717 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:15.102689    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c191734c-168e-4d2d-8127-0a824d243685" path="/var/lib/kubelet/pods/c191734c-168e-4d2d-8127-0a824d243685/volumes"
Jul 21 18:46:15.791901 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:15.791872    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7562-140036458792976"
Jul 21 18:46:15.792088 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:15.791936    2578 scope.go:117] "RemoveContainer" containerID="e6aabbfce17e515493769d4c2750f670540471f208dc882f9583150104e50543"
Jul 21 18:46:36.982144 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:36.982053    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"]
Jul 21 18:46:36.982752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:36.982325    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy" containerID="cri-o://c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e" gracePeriod=30
Jul 21 18:46:42.233197 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.233170    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:46:42.339049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.338967    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-credential-socket\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339003    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575e52f8-c943-4c6e-a8b2-220e83afb292-istiod-ca-cert\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339023    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-token\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339049 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339043    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-data\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339205    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25p7k\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-kube-api-access-25p7k\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339282    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:42.339361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339296    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-socket\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339361 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-envoy\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339575 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339396    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-podinfo\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339575 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339401    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/575e52f8-c943-4c6e-a8b2-220e83afb292-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:46:42.339575 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339458    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-certs\") pod \"575e52f8-c943-4c6e-a8b2-220e83afb292\" (UID: \"575e52f8-c943-4c6e-a8b2-220e83afb292\") "
Jul 21 18:46:42.339789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339584    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:42.339849 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339816    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.339849 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339843    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/575e52f8-c943-4c6e-a8b2-220e83afb292-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.339964 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339814    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:42.339964 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.339859    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.340073 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.340049    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-data" (OuterVolumeSpecName: "istio-data") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:42.341374 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.341346    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-token" (OuterVolumeSpecName: "istio-token") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:46:42.341374 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.341358    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:46:42.341522 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.341395    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-kube-api-access-25p7k" (OuterVolumeSpecName: "kube-api-access-25p7k") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "kube-api-access-25p7k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:46:42.341522 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.341470    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "575e52f8-c943-4c6e-a8b2-220e83afb292" (UID: "575e52f8-c943-4c6e-a8b2-220e83afb292"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:46:42.440271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.440242    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.440271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.440267    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.440271 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.440275    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.440491 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.440284    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.440491 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.440292    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/575e52f8-c943-4c6e-a8b2-220e83afb292-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.440491 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.440300    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25p7k\" (UniqueName: \"kubernetes.io/projected/575e52f8-c943-4c6e-a8b2-220e83afb292-kube-api-access-25p7k\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:46:42.919636 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.919578    2578 generic.go:356] "Generic (PLEG): container finished" podID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerID="c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e" exitCode=0
Jul 21 18:46:42.919636 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.919635    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"
Jul 21 18:46:42.919881 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.919660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" event={"ID":"575e52f8-c943-4c6e-a8b2-220e83afb292","Type":"ContainerDied","Data":"c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e"}
Jul 21 18:46:42.919881 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.919713    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx" event={"ID":"575e52f8-c943-4c6e-a8b2-220e83afb292","Type":"ContainerDied","Data":"95d72f97f29bc8a5e753cba711e86127fa95410603d2c49cd0c77db928b20be9"}
Jul 21 18:46:42.919881 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.919737    2578 scope.go:117] "RemoveContainer" containerID="c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e"
Jul 21 18:46:42.928721 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.928705    2578 scope.go:117] "RemoveContainer" containerID="3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4"
Jul 21 18:46:42.936877 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.936860    2578 scope.go:117] "RemoveContainer" containerID="c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e"
Jul 21 18:46:42.937118 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:46:42.937099    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e\": container with ID starting with c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e not found: ID does not exist" containerID="c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e"
Jul 21 18:46:42.937162 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.937128    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e"} err="failed to get container status \"c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e\": rpc error: code = NotFound desc = could not find container \"c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e\": container with ID starting with c5f8184aac9d8fda888d3d02bf395c52c6188a60bf739b2c44156b16b8ed407e not found: ID does not exist"
Jul 21 18:46:42.937162 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.937145    2578 scope.go:117] "RemoveContainer" containerID="3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4"
Jul 21 18:46:42.937381 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:46:42.937361    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4\": container with ID starting with 3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4 not found: ID does not exist" containerID="3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4"
Jul 21 18:46:42.937445 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.937391    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4"} err="failed to get container status \"3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4\": rpc error: code = NotFound desc = could not find container \"3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4\": container with ID starting with 3115416f54ee53da3de6bbc386c1e7515b8b910173acc25c9828b766e2eefed4 not found: ID does not exist"
Jul 21 18:46:42.944051 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.944027    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"]
Jul 21 18:46:42.948077 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:42.948057    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c1a1f4-openshift-default-84c8988859-mkjhx"]
Jul 21 18:46:43.102178 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:43.102148    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" path="/var/lib/kubelet/pods/575e52f8-c943-4c6e-a8b2-220e83afb292/volumes"
Jul 21 18:46:59.170959 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:59.170917    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"]
Jul 21 18:46:59.171428 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:46:59.171200    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy" containerID="cri-o://3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556" gracePeriod=30
Jul 21 18:47:04.424534 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.424503    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:47:04.538122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538041    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-credential-socket\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538079    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-envoy\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538094    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-certs\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538122 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538116    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a920794-bae8-4698-b6b2-1ae315ca3b80-istiod-ca-cert\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538162    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-token\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538187    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-data\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538222    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-podinfo\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538315    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4tsr\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-kube-api-access-f4tsr\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538339    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-socket\") pod \"1a920794-bae8-4698-b6b2-1ae315ca3b80\" (UID: \"1a920794-bae8-4698-b6b2-1ae315ca3b80\") "
Jul 21 18:47:04.538425 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538346    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:47:04.538762 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538449    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:47:04.538762 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538582    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:47:04.538762 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538665    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a920794-bae8-4698-b6b2-1ae315ca3b80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 18:47:04.539010 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.538973    2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a920794-bae8-4698-b6b2-1ae315ca3b80-istiod-ca-cert\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.539010 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.539005    2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.539129 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.539019    2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-credential-socket\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.539129 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.539032    2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-workload-certs\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.539223 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.539181    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-data" (OuterVolumeSpecName: "istio-data") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:47:04.540623 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.540574    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-token" (OuterVolumeSpecName: "istio-token") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:47:04.540623 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.540612    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 18:47:04.540789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.540720    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 18:47:04.540789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.540737    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-kube-api-access-f4tsr" (OuterVolumeSpecName: "kube-api-access-f4tsr") pod "1a920794-bae8-4698-b6b2-1ae315ca3b80" (UID: "1a920794-bae8-4698-b6b2-1ae315ca3b80"). InnerVolumeSpecName "kube-api-access-f4tsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 18:47:04.640451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.640421    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4tsr\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-kube-api-access-f4tsr\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.640451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.640447    2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-envoy\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.640451 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.640457    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-token\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.640687 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.640465    2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-data\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:04.640687 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:04.640473    2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a920794-bae8-4698-b6b2-1ae315ca3b80-istio-podinfo\") on node \"ip-10-0-140-73.ec2.internal\" DevicePath \"\""
Jul 21 18:47:05.022717 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.022682    2578 generic.go:356] "Generic (PLEG): container finished" podID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerID="3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556" exitCode=0
Jul 21 18:47:05.022888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.022730    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"
Jul 21 18:47:05.022888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.022770    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" event={"ID":"1a920794-bae8-4698-b6b2-1ae315ca3b80","Type":"ContainerDied","Data":"3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556"}
Jul 21 18:47:05.022888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.022843    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m" event={"ID":"1a920794-bae8-4698-b6b2-1ae315ca3b80","Type":"ContainerDied","Data":"860da994b09329496b90a98ec3cd1a6b6e83dec0065afcb7b9c40fbcdd21db3c"}
Jul 21 18:47:05.022888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.022865    2578 scope.go:117] "RemoveContainer" containerID="3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556"
Jul 21 18:47:05.032414 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.032395    2578 scope.go:117] "RemoveContainer" containerID="19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9"
Jul 21 18:47:05.040098 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.040084    2578 scope.go:117] "RemoveContainer" containerID="3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556"
Jul 21 18:47:05.040324 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:47:05.040306    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556\": container with ID starting with 3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556 not found: ID does not exist" containerID="3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556"
Jul 21 18:47:05.040400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.040337    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556"} err="failed to get container status \"3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556\": rpc error: code = NotFound desc = could not find container \"3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556\": container with ID starting with 3c1d559d076efd6e8d098b452764c55bf359489de9f9a13c93ade852439ca556 not found: ID does not exist"
Jul 21 18:47:05.040400 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.040361    2578 scope.go:117] "RemoveContainer" containerID="19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9"
Jul 21 18:47:05.040611 ip-10-0-140-73 kubenswrapper[2578]: E0721 18:47:05.040581    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9\": container with ID starting with 19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9 not found: ID does not exist" containerID="19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9"
Jul 21 18:47:05.040656 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.040611    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9"} err="failed to get container status \"19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9\": rpc error: code = NotFound desc = could not find container \"19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9\": container with ID starting with 19bf30919ceb41fded23467cc6da802c887a75f486eb55f5fea6678ce09b19b9 not found: ID does not exist"
Jul 21 18:47:05.046453 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.046433    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"]
Jul 21 18:47:05.050313 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.050290    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-0be774-openshift-default-6bcddf86c-w9c5m"]
Jul 21 18:47:05.102956 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.102921    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" path="/var/lib/kubelet/pods/1a920794-bae8-4698-b6b2-1ae315ca3b80/volumes"
Jul 21 18:47:05.398003 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.397980    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x87b7_23a33cf6-c769-4bb9-8a5f-ce716d059ed8/manager/0.log"
Jul 21 18:47:05.521719 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.521682    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cs9n8_b85bf252-4660-4a31-beda-038aab8cabc4/manager/0.log"
Jul 21 18:47:05.649668 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.649565    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d9f8c6488-fq4vk_52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf/manager/0.log"
Jul 21 18:47:05.764687 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.764657    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5trmw_d1f86d26-1b3d-4268-bfc8-6223e5167679/server/0.log"
Jul 21 18:47:05.893225 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:05.893196    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bfdvz_034f126e-24f6-42d4-95c3-5f63faea5d2e/manager/0.log"
Jul 21 18:47:06.271091 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:06.271066    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-v86px_b9640749-756b-4c6c-a645-6ad85dea8231/copy-manifests/0.log"
Jul 21 18:47:06.277939 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:06.277916    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-v86px_b9640749-756b-4c6c-a645-6ad85dea8231/manager/0.log"
Jul 21 18:47:06.625801 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:06.625723    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-75796d6899-fgmfx_9c4c96d5-a8e2-40fe-94a4-e1a4a47b7a5e/maas-api/0.log"
Jul 21 18:47:07.083910 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:07.083861    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-p6gtg_7b4e5056-e80f-407d-924c-fa0d40a5dd9a/postgres/0.log"
Jul 21 18:47:08.557866 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:08.557836    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-flk4v_634ddebd-0b55-4ce5-9f45-d8bc3e5c2626/manager/0.log"
Jul 21 18:47:08.670869 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:08.670840    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z4qbd_2a949a58-5882-44ed-8071-3326ea7d49bb/manager/0.log"
Jul 21 18:47:08.900740 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:08.900709    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-754jw_f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7/registry-server/0.log"
Jul 21 18:47:09.254093 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:09.253994    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5b8ss_7cbc79ee-68ce-4fdf-b495-15a4abf9a08b/manager/0.log"
Jul 21 18:47:09.612752 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:09.612661    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47227g_44bffe7e-2c75-4c93-98c7-1cca9b6818b7/istio-proxy/0.log"
Jul 21 18:47:09.841104 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:09.841048    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-7rb7f_d1856fec-efd0-4eae-b789-7675b6c48050/kube-auth-proxy/0.log"
Jul 21 18:47:10.083573 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:10.083537    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t5x67_0d30f709-825f-4f5c-9b68-da46b32a7d5b/istio-proxy/1.log"
Jul 21 18:47:10.786438 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:10.786398    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-slkpz_39224ca6-488e-4777-94c3-15c5a7bd64df/main/0.log"
Jul 21 18:47:11.132561 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:11.132507    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fzcvfl_5f9e95a6-47d6-42fb-8563-1e02e2b70bb4/main/0.log"
Jul 21 18:47:11.249505 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:11.249477    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-9lq9z_1e04577c-b70c-4755-80ee-1708a3169ca2/main/0.log"
Jul 21 18:47:16.613555 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613525    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gg7gx/must-gather-nzplj"]
Jul 21 18:47:16.613924 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613887    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy"
Jul 21 18:47:16.613924 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613900    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy"
Jul 21 18:47:16.613924 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613908    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy"
Jul 21 18:47:16.613924 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613914    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy"
Jul 21 18:47:16.613924 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613923    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c191734c-168e-4d2d-8127-0a824d243685" containerName="test-curl-7562-140036458792976"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613928    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c191734c-168e-4d2d-8127-0a824d243685" containerName="test-curl-7562-140036458792976"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613938    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.613944    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.614003    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.614009    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.614016    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.614021    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="575e52f8-c943-4c6e-a8b2-220e83afb292" containerName="istio-proxy"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.614028    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c191734c-168e-4d2d-8127-0a824d243685" containerName="test-curl-7562-140036458792976"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.614088    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy"
Jul 21 18:47:16.614101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.614095    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a920794-bae8-4698-b6b2-1ae315ca3b80" containerName="istio-proxy"
Jul 21 18:47:16.618716 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.618697    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:16.621665 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.621639    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg7gx\"/\"kube-root-ca.crt\""
Jul 21 18:47:16.622888 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.622871    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-gg7gx\"/\"default-dockercfg-pzzcs\""
Jul 21 18:47:16.622960 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.622906    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-gg7gx\"/\"openshift-service-ca.crt\""
Jul 21 18:47:16.629175 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.629151    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/must-gather-nzplj"]
Jul 21 18:47:16.744656 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.744623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4gt2\" (UniqueName: \"kubernetes.io/projected/b6e1a3c5-5814-41c9-9762-49f1dba0fcd5-kube-api-access-b4gt2\") pod \"must-gather-nzplj\" (UID: \"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5\") " pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:16.744818 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.744674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b6e1a3c5-5814-41c9-9762-49f1dba0fcd5-must-gather-output\") pod \"must-gather-nzplj\" (UID: \"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5\") " pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:16.845840 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.845806    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4gt2\" (UniqueName: \"kubernetes.io/projected/b6e1a3c5-5814-41c9-9762-49f1dba0fcd5-kube-api-access-b4gt2\") pod \"must-gather-nzplj\" (UID: \"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5\") " pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:16.845840 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.845845    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b6e1a3c5-5814-41c9-9762-49f1dba0fcd5-must-gather-output\") pod \"must-gather-nzplj\" (UID: \"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5\") " pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:16.846171 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.846155    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b6e1a3c5-5814-41c9-9762-49f1dba0fcd5-must-gather-output\") pod \"must-gather-nzplj\" (UID: \"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5\") " pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:16.854842 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.854822    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4gt2\" (UniqueName: \"kubernetes.io/projected/b6e1a3c5-5814-41c9-9762-49f1dba0fcd5-kube-api-access-b4gt2\") pod \"must-gather-nzplj\" (UID: \"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5\") " pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:16.928309 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:16.928289    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/must-gather-nzplj"
Jul 21 18:47:17.062775 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:17.062752    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/must-gather-nzplj"]
Jul 21 18:47:17.064862 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:47:17.064835    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb6e1a3c5_5814_41c9_9762_49f1dba0fcd5.slice/crio-145aa33e261d3662939475b20eebccc4f0b61c0f761f68d5fb54fc14b3a2ca67 WatchSource:0}: Error finding container 145aa33e261d3662939475b20eebccc4f0b61c0f761f68d5fb54fc14b3a2ca67: Status 404 returned error can't find the container with id 145aa33e261d3662939475b20eebccc4f0b61c0f761f68d5fb54fc14b3a2ca67
Jul 21 18:47:17.082437 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:17.082411    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/must-gather-nzplj" event={"ID":"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5","Type":"ContainerStarted","Data":"145aa33e261d3662939475b20eebccc4f0b61c0f761f68d5fb54fc14b3a2ca67"}
Jul 21 18:47:18.089631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:18.089521    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/must-gather-nzplj" event={"ID":"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5","Type":"ContainerStarted","Data":"1506c7e7a0cfd1919f1a5f0cac100cf359c135e030888b4ce08600acb8d99a8f"}
Jul 21 18:47:18.089631 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:18.089572    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/must-gather-nzplj" event={"ID":"b6e1a3c5-5814-41c9-9762-49f1dba0fcd5","Type":"ContainerStarted","Data":"09115ebaf78e715dbc6f92f34a986481f249a180b155764962cd8f88d842d855"}
Jul 21 18:47:18.107785 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:18.107733    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gg7gx/must-gather-nzplj" podStartSLOduration=1.344309566 podStartE2EDuration="2.107716747s" totalImagesPullingTime="763.407181ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 18:47:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:47:18.106544794 +0000 UTC m=+2826.364987974" watchObservedRunningTime="2026-07-21 18:47:18.107716747 +0000 UTC m=+2826.366159927"
Jul 21 18:47:19.511199 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:19.511166    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zqmhv_df1e655a-c0cc-4d13-8716-6a0fe968b90d/global-pull-secret-syncer/0.log"
Jul 21 18:47:19.641221 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:19.641190    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vss4j_e588e98b-3542-4fdd-afa9-fb8b1e3d787e/konnectivity-agent/0.log"
Jul 21 18:47:19.718461 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:19.718426    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-73.ec2.internal_fd118746fb461e4f1c5de51b2e36feed/haproxy/0.log"
Jul 21 18:47:24.310768 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:24.310732    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-flk4v_634ddebd-0b55-4ce5-9f45-d8bc3e5c2626/manager/0.log"
Jul 21 18:47:24.346735 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:24.346706    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-z4qbd_2a949a58-5882-44ed-8071-3326ea7d49bb/manager/0.log"
Jul 21 18:47:24.425402 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:24.425367    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-754jw_f82fde1e-10eb-4ed3-89f8-dfb152d6e4e7/registry-server/0.log"
Jul 21 18:47:24.712329 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:24.712290    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-5b8ss_7cbc79ee-68ce-4fdf-b495-15a4abf9a08b/manager/0.log"
Jul 21 18:47:26.665101 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:26.665066    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzcwv_c022413f-df43-42a6-83db-cda2c9d8536e/node-exporter/0.log"
Jul 21 18:47:26.684745 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:26.684715    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzcwv_c022413f-df43-42a6-83db-cda2c9d8536e/kube-rbac-proxy/0.log"
Jul 21 18:47:26.704838 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:26.704796    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-rzcwv_c022413f-df43-42a6-83db-cda2c9d8536e/init-textfile/0.log"
Jul 21 18:47:30.106165 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.106128    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"]
Jul 21 18:47:30.113158 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.113134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.132547 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.132518    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"]
Jul 21 18:47:30.226524 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.226251    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-podres\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.226524 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.226316    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-sys\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.226524 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.226334    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-lib-modules\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.226524 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.226364    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-proc\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.226524 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.226400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgrd4\" (UniqueName: \"kubernetes.io/projected/58d05412-0822-42e7-a669-529c7fd994dd-kube-api-access-xgrd4\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327542 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327508    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-podres\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327563    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-sys\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327604    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-lib-modules\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-proc\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgrd4\" (UniqueName: \"kubernetes.io/projected/58d05412-0822-42e7-a669-529c7fd994dd-kube-api-access-xgrd4\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327682    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-sys\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327734    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-podres\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.327769 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327749    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-proc\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.328141 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.327791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/58d05412-0822-42e7-a669-529c7fd994dd-lib-modules\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.338767 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.338744    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgrd4\" (UniqueName: \"kubernetes.io/projected/58d05412-0822-42e7-a669-529c7fd994dd-kube-api-access-xgrd4\") pod \"perf-node-gather-daemonset-nsc85\" (UID: \"58d05412-0822-42e7-a669-529c7fd994dd\") " pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.428243 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.427992    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:30.603640 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:30.603319    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"]
Jul 21 18:47:30.607137 ip-10-0-140-73 kubenswrapper[2578]: W0721 18:47:30.607088    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod58d05412_0822_42e7_a669_529c7fd994dd.slice/crio-1dd0a1ed48acc3db45e2a4dfc7befa8084f024c14603d81a749fc9ff1d7e9c6b WatchSource:0}: Error finding container 1dd0a1ed48acc3db45e2a4dfc7befa8084f024c14603d81a749fc9ff1d7e9c6b: Status 404 returned error can't find the container with id 1dd0a1ed48acc3db45e2a4dfc7befa8084f024c14603d81a749fc9ff1d7e9c6b
Jul 21 18:47:31.185182 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:31.185141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85" event={"ID":"58d05412-0822-42e7-a669-529c7fd994dd","Type":"ContainerStarted","Data":"e59363a1e5dc83b6107dc49c2c2796ddd2ec5702646b5ae951d3207c55eaaee7"}
Jul 21 18:47:31.185692 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:31.185211    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85" event={"ID":"58d05412-0822-42e7-a669-529c7fd994dd","Type":"ContainerStarted","Data":"1dd0a1ed48acc3db45e2a4dfc7befa8084f024c14603d81a749fc9ff1d7e9c6b"}
Jul 21 18:47:31.185692 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:31.185507    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:31.217651 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:31.217582    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85" podStartSLOduration=1.217563693 podStartE2EDuration="1.217563693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 18:47:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 18:47:31.216585103 +0000 UTC m=+2839.475028285" watchObservedRunningTime="2026-07-21 18:47:31.217563693 +0000 UTC m=+2839.476006874"
Jul 21 18:47:31.884879 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:31.884851    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cbplt_96003adc-b20f-487f-b330-358bdf2473d0/dns/0.log"
Jul 21 18:47:31.904810 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:31.904787    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cbplt_96003adc-b20f-487f-b330-358bdf2473d0/kube-rbac-proxy/0.log"
Jul 21 18:47:32.020682 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:32.020650    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2q8df_fdfba343-022f-434b-b69a-64677d0384c5/dns-node-resolver/0.log"
Jul 21 18:47:32.655079 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:32.655048    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-58c7b989d5-9k7cr_0ae323bc-9fce-4bf6-9789-7acc02dd1e11/registry/0.log"
Jul 21 18:47:32.735864 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:32.735829    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xpnv6_d858ae26-2013-4bb3-be07-621fa975f0f1/node-ca/0.log"
Jul 21 18:47:33.705574 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:33.705541    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47227g_44bffe7e-2c75-4c93-98c7-1cca9b6818b7/istio-proxy/0.log"
Jul 21 18:47:33.856229 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:33.856202    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-7rb7f_d1856fec-efd0-4eae-b789-7675b6c48050/kube-auth-proxy/0.log"
Jul 21 18:47:33.912140 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:33.912103    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t5x67_0d30f709-825f-4f5c-9b68-da46b32a7d5b/istio-proxy/0.log"
Jul 21 18:47:33.969632 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:33.969533    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t5x67_0d30f709-825f-4f5c-9b68-da46b32a7d5b/istio-proxy/1.log"
Jul 21 18:47:34.752800 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:34.752759    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sltvh_d508cf76-c00c-42e6-bd27-88879c52c149/serve-healthcheck-canary/0.log"
Jul 21 18:47:35.320423 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:35.320393    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnv29_f87c1304-0a50-4bf2-8b8f-c3e8cea362ae/kube-rbac-proxy/0.log"
Jul 21 18:47:35.341391 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:35.341355    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnv29_f87c1304-0a50-4bf2-8b8f-c3e8cea362ae/exporter/0.log"
Jul 21 18:47:35.364198 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:35.364163    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hnv29_f87c1304-0a50-4bf2-8b8f-c3e8cea362ae/extractor/0.log"
Jul 21 18:47:37.201448 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:37.201423    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-gg7gx/perf-node-gather-daemonset-nsc85"
Jul 21 18:47:37.519143 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:37.519040    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x87b7_23a33cf6-c769-4bb9-8a5f-ce716d059ed8/manager/0.log"
Jul 21 18:47:37.862724 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:37.862632    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-cs9n8_b85bf252-4660-4a31-beda-038aab8cabc4/manager/0.log"
Jul 21 18:47:38.081416 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:38.081365    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-d9f8c6488-fq4vk_52ad5a9f-8500-4452-aecd-4aaa2ccf9bdf/manager/0.log"
Jul 21 18:47:38.105331 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:38.105275    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-5trmw_d1f86d26-1b3d-4268-bfc8-6223e5167679/server/0.log"
Jul 21 18:47:38.224416 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:38.224365    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-bfdvz_034f126e-24f6-42d4-95c3-5f63faea5d2e/manager/0.log"
Jul 21 18:47:38.393441 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:38.393399    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-v86px_b9640749-756b-4c6c-a645-6ad85dea8231/manager/0.log"
Jul 21 18:47:38.413606 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:38.413565    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6b54c48d7b-v86px_b9640749-756b-4c6c-a645-6ad85dea8231/copy-manifests/0.log"
Jul 21 18:47:39.908948 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:39.908915    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47227g_44bffe7e-2c75-4c93-98c7-1cca9b6818b7/istio-proxy/0.log"
Jul 21 18:47:40.040663 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:40.040629    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-7rb7f_d1856fec-efd0-4eae-b789-7675b6c48050/kube-auth-proxy/0.log"
Jul 21 18:47:40.078366 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:40.078335    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t5x67_0d30f709-825f-4f5c-9b68-da46b32a7d5b/istio-proxy/0.log"
Jul 21 18:47:40.140418 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:40.140385    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t5x67_0d30f709-825f-4f5c-9b68-da46b32a7d5b/istio-proxy/1.log"
Jul 21 18:47:41.467789 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:41.467758    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47227g_44bffe7e-2c75-4c93-98c7-1cca9b6818b7/istio-proxy/0.log"
Jul 21 18:47:41.594045 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:41.594012    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f49f77f5f-7rb7f_d1856fec-efd0-4eae-b789-7675b6c48050/kube-auth-proxy/0.log"
Jul 21 18:47:41.638074 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:41.637117    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t5x67_0d30f709-825f-4f5c-9b68-da46b32a7d5b/istio-proxy/0.log"
Jul 21 18:47:41.691173 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:41.691144    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-t5x67_0d30f709-825f-4f5c-9b68-da46b32a7d5b/istio-proxy/1.log"
Jul 21 18:47:46.554832 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.554807    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p9m59_0710eca1-fe6f-4b08-a308-1dbe4e70619e/kube-multus-additional-cni-plugins/0.log"
Jul 21 18:47:46.578364 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.578338    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p9m59_0710eca1-fe6f-4b08-a308-1dbe4e70619e/egress-router-binary-copy/0.log"
Jul 21 18:47:46.597604 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.597566    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p9m59_0710eca1-fe6f-4b08-a308-1dbe4e70619e/cni-plugins/0.log"
Jul 21 18:47:46.617310 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.617281    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p9m59_0710eca1-fe6f-4b08-a308-1dbe4e70619e/bond-cni-plugin/0.log"
Jul 21 18:47:46.636088 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.636060    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p9m59_0710eca1-fe6f-4b08-a308-1dbe4e70619e/routeoverride-cni/0.log"
Jul 21 18:47:46.657039 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.657012    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p9m59_0710eca1-fe6f-4b08-a308-1dbe4e70619e/whereabouts-cni-bincopy/0.log"
Jul 21 18:47:46.677864 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.677839    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-p9m59_0710eca1-fe6f-4b08-a308-1dbe4e70619e/whereabouts-cni/0.log"
Jul 21 18:47:46.909146 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:46.909115    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cvv4k_de361996-b492-47df-8f47-24e14804473e/kube-multus/0.log"
Jul 21 18:47:47.047196 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:47.047167    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pkx8c_0ef3f2e2-9940-4e62-a294-673fb6176efc/network-metrics-daemon/0.log"
Jul 21 18:47:47.065791 ip-10-0-140-73 kubenswrapper[2578]: I0721 18:47:47.065768    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-pkx8c_0ef3f2e2-9940-4e62-a294-673fb6176efc/kube-rbac-proxy/0.log"