Jul 21 12:27:40.895750 ip-10-0-143-161 systemd[1]: Starting Kubernetes Kubelet...
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.315012    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323111    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323133    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323138    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323143    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323148    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323152    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323156    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:27:41.339842 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323160    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323164    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323168    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323173    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323179    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323183    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323187    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323191    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323195    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323199    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323203    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323207    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323210    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323214    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323218    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323222    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323228    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323234    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323238    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323250    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:27:41.340978 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323255    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323259    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323263    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323270    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323275    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323280    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323284    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323289    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323293    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323298    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323302    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323306    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323310    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323314    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323318    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323322    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323326    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323329    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323333    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:27:41.341547 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323337    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323341    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323346    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323349    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323353    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323357    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323361    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323365    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323369    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323373    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323377    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323381    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323384    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323388    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323393    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323397    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323401    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323405    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323409    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323413    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:27:41.342067 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323418    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323422    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323425    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323429    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323433    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323452    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323456    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323460    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323464    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323469    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323473    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323477    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323480    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323484    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323490    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323495    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323499    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323504    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323508    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:27:41.342905 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323513    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323517    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323521    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323524    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323528    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323532    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323535    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323539    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323543    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323548    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323552    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323556    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323559    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.323563    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324624    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324633    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324637    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324641    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324646    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324650    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:27:41.343469 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324654    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324657    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324661    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324669    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324674    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324679    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324683    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324688    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324692    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324696    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324700    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324704    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324708    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324711    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324715    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324719    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324723    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324727    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324730    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:27:41.344054 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324735    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324739    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324744    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324747    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324751    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324755    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324759    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324762    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324767    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324770    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324774    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324778    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324782    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324787    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324790    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324794    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324798    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324802    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324806    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324809    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:27:41.344744 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324813    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324816    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324820    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324823    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324827    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324830    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324834    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324838    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324842    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324845    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324849    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324852    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324855    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324859    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324864    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324868    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324874    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324878    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324883    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324887    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:27:41.345351 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324891    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324895    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324899    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324903    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324907    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324911    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324916    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324920    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324923    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324927    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324931    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324934    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324938    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324942    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324945    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324949    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324953    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324956    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324959    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324963    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:27:41.436218 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324966    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:27:41.400089 ip-10-0-143-161 systemd[1]: Started Kubernetes Kubelet.
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324969    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324973    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324977    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324981    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324987    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324990    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324994    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.324997    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325001    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325005    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325009    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325012    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325016    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325606    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325614    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325618    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325622    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325626    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325630    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:27:41.437455 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325635    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325639    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325643    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325648    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325651    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325655    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325658    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325662    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325666    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325671    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325675    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325678    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325686    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325690    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325694    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325698    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325701    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325706    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325710    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325714    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325718    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:27:41.439971 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325722    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325726    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325729    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325733    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325737    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325740    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325744    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325747    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325751    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325755    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325759    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325763    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325766    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325770    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325777    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325781    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325785    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325788    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325792    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325796    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:27:41.441039 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325799    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325803    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325806    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325815    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325818    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325822    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325826    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325829    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325833    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325837    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325840    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325844    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325848    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325851    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325855    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325859    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325862    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325866    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325870    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:27:42.780390 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325873    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325877    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325881    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325886    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325890    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325894    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325897    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325903    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325907    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325911    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325915    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325921    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325925    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325930    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325935    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325938    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325944    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325948    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325952    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:27:42.782311 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325956    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325962    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325967    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325970    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325973    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325977    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325981    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325985    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325989    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325993    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.325996    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.326000    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.326004    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.326008    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326089    2577 flags.go:64] FLAG: --address="0.0.0.0"
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326099    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326115    2577 flags.go:64] FLAG: --anonymous-auth="true"
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326121    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326127    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326132    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326139    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 21 12:27:42.783393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326145    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326150    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326155    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326161    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326165    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326170    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326174    2577 flags.go:64] FLAG: --cgroup-root=""
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326178    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326184    2577 flags.go:64] FLAG: --client-ca-file=""
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326188    2577 flags.go:64] FLAG: --cloud-provider="external"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326193    2577 flags.go:64] FLAG: --cluster-dns="[]"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326199    2577 flags.go:64] FLAG: --cluster-domain=""
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326204    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326208    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326213    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326218    2577 flags.go:64] FLAG: --container-log-max-files="5"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326224    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326228    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326233    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326238    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326242    2577 flags.go:64] FLAG: --contention-profiling="false"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326247    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326251    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326256    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 21 12:27:42.785051 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326260    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326266    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326270    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326274    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326279    2577 flags.go:64] FLAG: --enable-load-reader="false"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326283    2577 flags.go:64] FLAG: --enable-server="true"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326288    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326296    2577 flags.go:64] FLAG: --event-burst="100"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326301    2577 flags.go:64] FLAG: --event-qps="50"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326305    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326310    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326314    2577 flags.go:64] FLAG: --eviction-hard=""
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326319    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326324    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326328    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326333    2577 flags.go:64] FLAG: --eviction-soft=""
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326339    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326343    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326348    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326352    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326356    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326360    2577 flags.go:64] FLAG: --fail-swap-on="true"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326364    2577 flags.go:64] FLAG: --feature-gates=""
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326370    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326374    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 21 12:27:42.786308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326379    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326384    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326388    2577 flags.go:64] FLAG: --healthz-port="10248"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326393    2577 flags.go:64] FLAG: --help="false"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326397    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-143-161.ec2.internal"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326402    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326406    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326410    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326415    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326421    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326425    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326429    2577 flags.go:64] FLAG: --image-service-endpoint=""
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326433    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326452    2577 flags.go:64] FLAG: --kube-api-burst="100"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326459    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326463    2577 flags.go:64] FLAG: --kube-api-qps="50"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326468    2577 flags.go:64] FLAG: --kube-reserved=""
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326472    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326478    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326483    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326487    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326491    2577 flags.go:64] FLAG: --lock-file=""
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326496    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326502    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 21 12:27:42.787803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326506    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326514    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326519    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326523    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326527    2577 flags.go:64] FLAG: --logging-format="text"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326532    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326536    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326541    2577 flags.go:64] FLAG: --manifest-url=""
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326545    2577 flags.go:64] FLAG: --manifest-url-header=""
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326552    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326556    2577 flags.go:64] FLAG: --max-open-files="1000000"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326562    2577 flags.go:64] FLAG: --max-pods="110"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326567    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326571    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326576    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326580    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326587    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326591    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326595    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326612    2577 flags.go:64] FLAG: --node-status-max-images="50"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326617    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326621    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326631    2577 flags.go:64] FLAG: --pod-cidr=""
Jul 21 12:27:42.789090 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326635    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326642    2577 flags.go:64] FLAG: --pod-manifest-path=""
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326646    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326651    2577 flags.go:64] FLAG: --pods-per-core="0"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326656    2577 flags.go:64] FLAG: --port="10250"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326660    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326665    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01b9317418f018bf2"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326670    2577 flags.go:64] FLAG: --qos-reserved=""
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326674    2577 flags.go:64] FLAG: --read-only-port="10255"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326680    2577 flags.go:64] FLAG: --register-node="true"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326684    2577 flags.go:64] FLAG: --register-with-taints=""
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326690    2577 flags.go:64] FLAG: --registry-burst="10"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326693    2577 flags.go:64] FLAG: --registry-qps="5"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326698    2577 flags.go:64] FLAG: --reserved-cpus=""
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326703    2577 flags.go:64] FLAG: --reserved-memory=""
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326708    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326713    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326718    2577 flags.go:64] FLAG: --rotate-certificates="false"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326722    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326726    2577 flags.go:64] FLAG: --runonce="false"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326730    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326734    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326739    2577 flags.go:64] FLAG: --seccomp-default="false"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326743    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326747    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326751    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 21 12:27:42.789918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326756    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326760    2577 flags.go:64] FLAG: --storage-driver-password="root"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326765    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326769    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326773    2577 flags.go:64] FLAG: --storage-driver-user="root"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326779    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326784    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326789    2577 flags.go:64] FLAG: --system-cgroups=""
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326793    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326801    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326806    2577 flags.go:64] FLAG: --tls-cert-file=""
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326814    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326820    2577 flags.go:64] FLAG: --tls-min-version=""
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326825    2577 flags.go:64] FLAG: --tls-private-key-file=""
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326829    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326834    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326838    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326843    2577 flags.go:64] FLAG: --v="2"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326848    2577 flags.go:64] FLAG: --version="false"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326854    2577 flags.go:64] FLAG: --vmodule=""
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326860    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.326865    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327085    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327093    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:27:42.790913 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327099    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327104    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327109    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327113    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327117    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327122    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327126    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327130    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327135    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327139    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327143    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327147    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327151    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327157    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327161    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327164    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327168    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327172    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327176    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327180    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:27:42.791677 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327190    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327195    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327199    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327206    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327210    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327214    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327218    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327222    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327226    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327230    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327234    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327237    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327241    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327245    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327249    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327253    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327257    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327260    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327264    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327268    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:27:42.792486 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327271    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327275    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327279    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327282    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327286    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327292    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327295    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327299    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327305    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327309    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327313    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327318    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327322    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327334    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327340    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327345    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327349    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327353    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327357    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:27:42.793251 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327360    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327364    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327368    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327372    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327375    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327379    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327383    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327387    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327391    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327394    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327398    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327402    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327405    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327410    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327413    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327417    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327421    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327425    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327431    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327448    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:27:42.793988 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327452    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327456    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327460    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327464    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327468    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327472    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327476    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327486    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327492    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327496    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327500    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327504    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327508    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327512    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327515    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327519    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327523    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:27:42.794721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.327526    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.328294    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.335705    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.335731    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.336633    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.336644    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336734    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336742    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336746    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336749    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336756    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336759    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336765    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336768    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336771    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:27:42.795360 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336777    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336780    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336783    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336788    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336791    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336793    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336796    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336802    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336805    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336808    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336813    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336817    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336819    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336824    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336827    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336830    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336832    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336837    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336840    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336843    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:27:42.796042 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336846    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336850    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336853    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336855    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336861    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336863    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336866    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336869    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336874    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336876    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336879    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336882    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336887    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336889    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336892    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336896    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336899    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336902    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336904    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336907    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:27:42.796773 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336910    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336912    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336915    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336924    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336928    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336931    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336934    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336936    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336941    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336944    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336946    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336949    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336954    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336956    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336959    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336961    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336968    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336972    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336977    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336981    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:27:42.797529 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336989    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336992    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336995    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.336998    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337004    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337007    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337009    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337014    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337017    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337020    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337022    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337027    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337030    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337033    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337037    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337042    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337046    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337052    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337055    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:27:42.798269 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337058    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337064    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337068    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337071    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337076    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337079    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337081    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337086    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337089    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337091    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337093    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.337101    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337249    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337254    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 21 12:27:42.798981 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337259    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337261    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337264    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337269    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337272    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337276    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337282    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337285    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337288    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337293    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337296    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337299    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337303    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337307    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337309    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337315    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337318    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337320    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337323    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 21 12:27:42.799525 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337326    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337328    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337331    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337336    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337338    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337341    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337346    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337348    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337351    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337353    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337358    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337361    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337364    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337366    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337370    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337374    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337381    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337384    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337387    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337392    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 21 12:27:42.800241 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337395    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337398    2577 feature_gate.go:328] unrecognized feature gate: Example
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337403    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337406    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337409    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337412    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337416    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337419    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337423    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337428    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337430    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337433    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337457    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337463    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337467    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337472    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337475    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337477    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337480    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337485    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 21 12:27:42.801034 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337488    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337490    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337495    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337498    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337500    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337505    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337508    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337511    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337516    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337519    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337521    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337524    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337526    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337529    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337531    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337534    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337538    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337541    2577 feature_gate.go:328] unrecognized feature gate: Example2
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337544    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 21 12:27:42.801785 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337548    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337554    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337557    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337562    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337564    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337567    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337571    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337574    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337576    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337579    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337584    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337586    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337589    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337591    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337596    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337599    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337601    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337605    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 21 12:27:42.802499 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:41.337608    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.337613    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.337768    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.340363    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.342074    2577 server.go:1019] "Starting client certificate rotation"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.342194    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.342241    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.345741    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.347836    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.360743    2577 log.go:25] "Validated CRI v1 runtime API"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.362968    2577 log.go:25] "Validated CRI v1 image API"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.367007    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 21 12:27:42.803145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.369827    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8bd07a70-5925-4cc0-9a81-09fa938257f3:/dev/nvme0n1p4 de5c5e65-f6df-4ea4-b84e-6965907f4c73:/dev/nvme0n1p3]
Jul 21 12:27:42.803519 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.369851    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 21 12:27:42.803519 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.370985    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.376574    2577 manager.go:216] Machine: {Timestamp:2026-07-21 12:27:41.374929591 +0000 UTC m=+0.360462869 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101172 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec240e3a8d26389881ea58e7bdfa133c SystemUUID:ec240e3a-8d26-3898-81ea-58e7bdfa133c BootID:b1f04262-501c-4c48-a0a7-38f1d8627a45 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f4:59:4d:60:7f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f4:59:4d:60:7f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:00:9e:75:d7:57 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.376720    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.376830    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.380186    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.380214    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-143-161.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.380974    2577 topology_manager.go:138] "Creating topology manager with none policy"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.380984    2577 container_manager_linux.go:309] "Creating device plugin manager"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.380991    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.381557    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.383701    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.386949    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387089    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387119    2577 kubelet.go:486] "Attempting to sync node with API server"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387133    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387148    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387158    2577 kubelet.go:398] "Adding apiserver pod source"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387168    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 21 12:27:42.803591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387568    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.387585    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.390613    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.392726    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.392761    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393620    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393635    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393642    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393647    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393657    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393667    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393676    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393682    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393698    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393704    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393712    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393719    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.393947    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wgfgr"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.395362    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.395383    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.396052    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.396081    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-143-161.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.399201    2577 server.go:1268] "Started kubelet"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.399244    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.399835    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.401384    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.401754    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wgfgr"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.405580    2577 server.go:310] "Adding debug handlers to kubelet server"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.405738    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.406751    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.410132    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-143-161.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.410155    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-143-161.ec2.internal.18c44dc520c02dcb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-143-161.ec2.internal,UID:ip-10-0-143-161.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-143-161.ec2.internal,},FirstTimestamp:2026-07-21 12:27:41.398920651 +0000 UTC m=+0.384453908,LastTimestamp:2026-07-21 12:27:41.398920651 +0000 UTC m=+0.384453908,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-143-161.ec2.internal,}"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.413121    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.413137    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.413185    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.413867    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.413882    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414007    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414019    2577 factory.go:55] Registering systemd factory
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414033    2577 factory.go:223] Registration of the systemd container factory successfully
Jul 21 12:27:42.804170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414034    2577 reconstruct.go:97] "Volume reconstruction finished"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414045    2577 reconciler.go:29] "Reconciler: start to sync state"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414246    2577 factory.go:153] Registering CRI-O factory
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414258    2577 factory.go:223] Registration of the crio container factory successfully
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414309    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414333    2577 factory.go:103] Registering Raw factory
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414347    2577 manager.go:1196] Started watching for new ooms in manager
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.414811    2577 manager.go:349] Starting recovery of all containers
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.422701    2577 manager.go:354] Recovery completed
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.430077    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.430884    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.446697    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-143-161.ec2.internal\" not found" node="ip-10-0-143-161.ec2.internal"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.446779    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.446809    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.446820    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.447398    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.447406    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.447407    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.447473    2577 state_mem.go:36] "Initialized new in-memory state store"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.448397    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.448420    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.448430    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.448916    2577 policy_none.go:49] "None policy: Start"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.448928    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.448939    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.450052    2577 policy_none.go:47] "Start"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.476024    2577 manager.go:337] "Starting Device Plugin manager"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.476053    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.476062    2577 server.go:87] "Starting device plugin registration server"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.476309    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.476319    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.476420    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.476524    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:41.476535    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.476943    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:41.476990    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.777163    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.779006    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.779039    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.779055    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.779094    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-143-161.ec2.internal"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.790052    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-143-161.ec2.internal"
Jul 21 12:27:42.805441 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:42.790210    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-143-161.ec2.internal\": node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:42.814414 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:42.814394    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:42.824453 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.824419    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 21 12:27:42.825829 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.825807    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 21 12:27:42.825914 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.825849    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 21 12:27:42.825914 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.825898    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 21 12:27:42.826000 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:42.825941    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 21 12:27:42.828471 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.828449    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:27:42.914971 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:42.914932    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:42.926199 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.926153    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"]
Jul 21 12:27:42.926323 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.926252    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:27:42.927210 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.927192    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:27:42.927302 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.927229    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:27:42.927302 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.927245    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:27:42.928667 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.928651    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:27:42.928802 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.928786    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:42.928841 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.928819    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:27:42.929436 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.929420    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:27:42.929436 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.929434    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:27:42.929566 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.929450    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:27:42.929566 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.929455    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:27:42.929566 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.929463    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:27:42.929566 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.929464    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:27:42.930827 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.930811    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:42.930905 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.930843    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 21 12:27:42.931545 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.931531    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientMemory"
Jul 21 12:27:42.931629 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.931560    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasNoDiskPressure"
Jul 21 12:27:42.931629 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:42.931574    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeHasSufficientPID"
Jul 21 12:27:42.954764 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:42.954735    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-161.ec2.internal\" not found" node="ip-10-0-143-161.ec2.internal"
Jul 21 12:27:42.958419 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:42.958404    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-143-161.ec2.internal\" not found" node="ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.015133 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.015050    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.016209 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.016186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4835d46be4ce0f9c6f53c4a51eee7a04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"4835d46be4ce0f9c6f53c4a51eee7a04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.016330 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.016215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4823430b3b83d7b0cb59efe2028dc586-config\") pod \"kube-apiserver-proxy-ip-10-0-143-161.ec2.internal\" (UID: \"4823430b3b83d7b0cb59efe2028dc586\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.016330 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.016235    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4835d46be4ce0f9c6f53c4a51eee7a04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"4835d46be4ce0f9c6f53c4a51eee7a04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.116126 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.116084    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.117227 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.117200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4835d46be4ce0f9c6f53c4a51eee7a04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"4835d46be4ce0f9c6f53c4a51eee7a04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.117334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.117241    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4835d46be4ce0f9c6f53c4a51eee7a04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"4835d46be4ce0f9c6f53c4a51eee7a04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.117334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.117261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4823430b3b83d7b0cb59efe2028dc586-config\") pod \"kube-apiserver-proxy-ip-10-0-143-161.ec2.internal\" (UID: \"4823430b3b83d7b0cb59efe2028dc586\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.117334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.117308    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/4823430b3b83d7b0cb59efe2028dc586-config\") pod \"kube-apiserver-proxy-ip-10-0-143-161.ec2.internal\" (UID: \"4823430b3b83d7b0cb59efe2028dc586\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.117334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.117310    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4835d46be4ce0f9c6f53c4a51eee7a04-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"4835d46be4ce0f9c6f53c4a51eee7a04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.117334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.117310    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4835d46be4ce0f9c6f53c4a51eee7a04-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal\" (UID: \"4835d46be4ce0f9c6f53c4a51eee7a04\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.216557 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.216517    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.256816 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.256781    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.261874 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.261856    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:43.317036 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.316937    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.417505 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.417459    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.517948 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.517908    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.542187 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.542164    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 21 12:27:43.542575 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.542310    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 12:27:43.542575 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.542342    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 21 12:27:43.610721 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.610624    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 12:22:42 +0000 UTC" deadline="2028-04-05 11:49:21.517256724 +0000 UTC"
Jul 21 12:27:43.610721 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.610664    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14975h21m37.90659563s"
Jul 21 12:27:43.618389 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.618365    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.718827 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.718795    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.813830 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:43.813793    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4835d46be4ce0f9c6f53c4a51eee7a04.slice/crio-802f0f4daa19f0bc33d43ce2cc672091f8424f00345ccdd5a0b19004b1bae915 WatchSource:0}: Error finding container 802f0f4daa19f0bc33d43ce2cc672091f8424f00345ccdd5a0b19004b1bae915: Status 404 returned error can't find the container with id 802f0f4daa19f0bc33d43ce2cc672091f8424f00345ccdd5a0b19004b1bae915
Jul 21 12:27:43.814189 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:43.814171    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4823430b3b83d7b0cb59efe2028dc586.slice/crio-910548c42d90aff73e25b4fdf053435bf0408e63610e99b6e6ac89dd54b01111 WatchSource:0}: Error finding container 910548c42d90aff73e25b4fdf053435bf0408e63610e99b6e6ac89dd54b01111: Status 404 returned error can't find the container with id 910548c42d90aff73e25b4fdf053435bf0408e63610e99b6e6ac89dd54b01111
Jul 21 12:27:43.818331 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.818312    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:27:43.818904 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.818887    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.829533 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.829480    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" event={"ID":"4823430b3b83d7b0cb59efe2028dc586","Type":"ContainerStarted","Data":"910548c42d90aff73e25b4fdf053435bf0408e63610e99b6e6ac89dd54b01111"}
Jul 21 12:27:43.830652 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.830625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" event={"ID":"4835d46be4ce0f9c6f53c4a51eee7a04","Type":"ContainerStarted","Data":"802f0f4daa19f0bc33d43ce2cc672091f8424f00345ccdd5a0b19004b1bae915"}
Jul 21 12:27:43.920074 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:43.919986    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:43.955470 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:43.955441    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:27:44.020571 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.020541    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:44.121242 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.121207    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-143-161.ec2.internal\" not found"
Jul 21 12:27:44.185948 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.185877    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:27:44.214803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.214776    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:44.225412 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.225389    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:27:44.226563 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.226551    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal"
Jul 21 12:27:44.232863 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.232838    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 21 12:27:44.588777 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.588744    2577 apiserver.go:52] "Watching apiserver"
Jul 21 12:27:44.597817 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.597788    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 21 12:27:44.598346 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.598318    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp","openshift-dns/node-resolver-76zlc","openshift-multus/multus-additional-cni-plugins-c5fnk","openshift-multus/multus-qff6d","openshift-ovn-kubernetes/ovnkube-node-lqpqc","kube-system/global-pull-secret-syncer-h9wst","kube-system/konnectivity-agent-z5xzk","openshift-cluster-node-tuning-operator/tuned-rplnb","openshift-image-registry/node-ca-zsqm9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal","openshift-multus/network-metrics-daemon-kmkdn","openshift-network-diagnostics/network-check-target-dztrl","openshift-network-operator/iptables-alerter-k4tt7"]
Jul 21 12:27:44.600042 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.600017    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.600137 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.600097    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:44.601951 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.601928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:44.602063 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.602021    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:44.603499 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.603479    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.605716 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.605690    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wdqxt\""
Jul 21 12:27:44.606568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.605931    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 21 12:27:44.606568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.606178    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 21 12:27:44.606568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.606546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.608468 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.608431    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.609215 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.609139    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 21 12:27:44.609330 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.609282    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 21 12:27:44.609397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.609346    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 21 12:27:44.609397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.609371    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-b7dcf\""
Jul 21 12:27:44.609516 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.609480    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 21 12:27:44.609686 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.609655    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 21 12:27:44.610314 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.609905    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.610558 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.610528    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 21 12:27:44.610807 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.610787    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-l4wzx\""
Jul 21 12:27:44.611498 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.611478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.612568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.612545    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 21 12:27:44.612989 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.612765    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 21 12:27:44.612989 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.612830    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 21 12:27:44.612989 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.612845    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 21 12:27:44.612989 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.612765    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vqdx7\""
Jul 21 12:27:44.613235 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.613121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.614062 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.613545    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 21 12:27:44.614062 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.613618    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 21 12:27:44.614062 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.613780    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 21 12:27:44.614062 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.613830    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 21 12:27:44.614062 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.613881    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:27:44.614358 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.614122    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-774nl\""
Jul 21 12:27:44.616149 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.615259    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:27:44.616149 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.615455    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 21 12:27:44.616149 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.615473    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8tnml\""
Jul 21 12:27:44.616149 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.615690    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 21 12:27:44.616149 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.615805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.618096 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.617870    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 21 12:27:44.618915 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.618889    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 21 12:27:44.620370 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.619199    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-xmszp\""
Jul 21 12:27:44.620370 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.619452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.620370 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.620253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 21 12:27:44.622109 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.621888    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 21 12:27:44.623079 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.622542    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 21 12:27:44.623079 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.622580    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f869m\""
Jul 21 12:27:44.623079 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.622799    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 21 12:27:44.623751 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.623721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c395bef-58b3-45ef-b677-db43aa258be1-tmp\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.623893 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.623875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-device-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.624023 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.624009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-sys-fs\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.624147 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.624120    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-socket-dir-parent\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.625528 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.625509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-kubelet\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.626168 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.625241    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:44.626411 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.625693    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-systemd\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.626779 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.626764    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-node-log\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.626900 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.626884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-cni-bin\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.627025 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627006    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-lib-modules\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.627123 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqfhc\" (UniqueName: \"kubernetes.io/projected/7c395bef-58b3-45ef-b677-db43aa258be1-kube-api-access-qqfhc\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.627210 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-kubelet-config\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.627303 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-dbus\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.627394 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627381    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsqxg\" (UniqueName: \"kubernetes.io/projected/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-kube-api-access-wsqxg\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.627494 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-ovn\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.627581 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.627665 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627654    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-slash\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.627742 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-env-overrides\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.627829 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-hostroot\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.627916 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.627904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:44.628084 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-var-lib-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.628212 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628188    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8347e5af-bb13-45e0-9025-b43410d820b8-ovn-node-metrics-cert\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.628299 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-modprobe-d\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.628394 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-k8s-cni-cncf-io\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.628471 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628204    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g7qtq\""
Jul 21 12:27:44.628555 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-conf-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.628649 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2514581-9d70-45d3-8cb2-3d6e0734d559-host-slash\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.628746 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628732    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-tmp-dir\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.628834 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628822    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-cnibin\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.628924 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-cni-netd\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.629009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628333    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 21 12:27:44.629236 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.626455    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:44.629390 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.629371    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:44.629473 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628260    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 21 12:27:44.629657 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-kubernetes\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.629743 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysctl-d\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.629743 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-run\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.629743 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-socket-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.629743 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.628389    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 21 12:27:44.629949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8395345-90a8-48e4-8fc1-aa9545d0a98a-cni-binary-copy\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.629949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-daemon-config\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.629949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-ovnkube-config\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.629949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-ovnkube-script-lib\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.629949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-multus-certs\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.629949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629886    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.629949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.629995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-netns\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630022    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2514581-9d70-45d3-8cb2-3d6e0734d559-iptables-alerter-script\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc9gk\" (UniqueName: \"kubernetes.io/projected/8347e5af-bb13-45e0-9025-b43410d820b8-kube-api-access-gc9gk\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630068    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-etc-kubernetes\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630098    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630129    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-hosts-file\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-kubelet\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-log-socket\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-host\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630269    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l52ww\" (UniqueName: \"kubernetes.io/projected/bc6e6606-0356-41e9-b18d-a646137ee771-kube-api-access-l52ww\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-cnibin\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-etc-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.630364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630359    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-etc-selinux\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630385    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-system-cni-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6trb\" (UniqueName: \"kubernetes.io/projected/c8395345-90a8-48e4-8fc1-aa9545d0a98a-kube-api-access-t6trb\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-os-release\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630473    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-system-cni-dir\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630498    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-run-netns\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-var-lib-kubelet\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630569    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-registration-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-os-release\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630614    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-systemd-units\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-cni-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-cni-multus\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630742    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-cni-binary-copy\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.631053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.630770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7wwp\" (UniqueName: \"kubernetes.io/projected/eab09017-f4c7-4571-854c-ea23e76f8dfe-kube-api-access-c7wwp\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysctl-conf\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-systemd\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c395bef-58b3-45ef-b677-db43aa258be1-etc-tuned\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-cni-bin\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631583    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njwmp\" (UniqueName: \"kubernetes.io/projected/e2514581-9d70-45d3-8cb2-3d6e0734d559-kube-api-access-njwmp\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysconfig\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.631840 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.631816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-sys\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.650693 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.650662    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-f67kj"
Jul 21 12:27:44.657379 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.657164    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-f67kj"
Jul 21 12:27:44.716084 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.716042    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 21 12:27:44.732005 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.731947    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-etc-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.732005 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.731999    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-etc-selinux\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732029    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb6k8\" (UniqueName: \"kubernetes.io/projected/9f6ab4a5-3a99-439c-934c-e07efea84081-kube-api-access-qb6k8\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732052    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-system-cni-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6trb\" (UniqueName: \"kubernetes.io/projected/c8395345-90a8-48e4-8fc1-aa9545d0a98a-kube-api-access-t6trb\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-os-release\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-system-cni-dir\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-run-netns\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732166    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-var-lib-kubelet\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-registration-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.732243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-os-release\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-systemd-units\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-cni-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-cni-multus\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732358    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-cni-binary-copy\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7wwp\" (UniqueName: \"kubernetes.io/projected/eab09017-f4c7-4571-854c-ea23e76f8dfe-kube-api-access-c7wwp\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysctl-conf\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732432    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-systemd\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732501    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-var-lib-kubelet\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732566    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-os-release\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-registration-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732615    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-etc-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732616    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-systemd-units\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732651    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-etc-selinux\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732765    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-system-cni-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.732839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-system-cni-dir\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.733586 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732847    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-cni-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.733586 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-systemd\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.733586 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-run-netns\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.733586 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c395bef-58b3-45ef-b677-db43aa258be1-etc-tuned\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.733586 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.732995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-cni-bin\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.733811 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.733702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njwmp\" (UniqueName: \"kubernetes.io/projected/e2514581-9d70-45d3-8cb2-3d6e0734d559-kube-api-access-njwmp\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.733811 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.733736    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.733811 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.733777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.734004 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.733811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.734004 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.733834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysconfig\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.734004 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.733859    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-sys\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.734151 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-cni-bin\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.734200 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734148    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 21 12:27:44.734273 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-cni-multus\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-cni-binary-copy\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysctl-conf\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734471    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-os-release\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysconfig\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-run-ovn-kubernetes\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.734941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-sys\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735094    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.735134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735105    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-tuning-conf-dir\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.735682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c395bef-58b3-45ef-b677-db43aa258be1-tmp\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.735682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-device-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.735682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-sys-fs\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.735682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735463    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-socket-dir-parent\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.735682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-kubelet\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.735682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-sys-fs\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.735682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735651    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-device-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-systemd\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-socket-dir-parent\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-node-log\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735756    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-var-lib-kubelet\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-cni-bin\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-systemd\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735820    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-lib-modules\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqfhc\" (UniqueName: \"kubernetes.io/projected/7c395bef-58b3-45ef-b677-db43aa258be1-kube-api-access-qqfhc\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735866    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-node-log\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-kubelet-config\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735971    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-lib-modules\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.735910    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-dbus\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736033    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsqxg\" (UniqueName: \"kubernetes.io/projected/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-kube-api-access-wsqxg\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-ovn\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736243    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-cni-bin\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9f6ab4a5-3a99-439c-934c-e07efea84081-serviceca\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.736920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736312    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-kubelet-config\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/157a0b56-df6f-4a64-800a-84d8a7596eb9-konnectivity-ca\") pod \"konnectivity-agent-z5xzk\" (UID: \"157a0b56-df6f-4a64-800a-84d8a7596eb9\") " pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736343    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-dbus\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736357    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-run-ovn\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736363    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-kubelet-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736620    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-slash\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-env-overrides\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736698    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-slash\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736700    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-hostroot\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736808    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/157a0b56-df6f-4a64-800a-84d8a7596eb9-agent-certs\") pod \"konnectivity-agent-z5xzk\" (UID: \"157a0b56-df6f-4a64-800a-84d8a7596eb9\") " pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-var-lib-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736903    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8347e5af-bb13-45e0-9025-b43410d820b8-ovn-node-metrics-cert\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-modprobe-d\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.736985    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-k8s-cni-cncf-io\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-conf-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.737699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2514581-9d70-45d3-8cb2-3d6e0734d559-host-slash\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-tmp-dir\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737113    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-var-lib-openvswitch\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-hostroot\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737174    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-env-overrides\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-cnibin\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-cni-netd\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737247    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-kubernetes\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-conf-dir\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-kubernetes\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2514581-9d70-45d3-8cb2-3d6e0734d559-host-slash\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysctl-d\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737539    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-run\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737562    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-socket-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8395345-90a8-48e4-8fc1-aa9545d0a98a-cni-binary-copy\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-daemon-config\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fhwr\" (UniqueName: \"kubernetes.io/projected/94d09ea1-66fe-4ee3-95ef-65866d92821c-kube-api-access-2fhwr\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-tmp-dir\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.738386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737684    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-ovnkube-config\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-ovnkube-script-lib\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737810    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-multus-certs\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737842    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737877    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-netns\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2514581-9d70-45d3-8cb2-3d6e0734d559-iptables-alerter-script\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.737987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f6ab4a5-3a99-439c-934c-e07efea84081-host\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc9gk\" (UniqueName: \"kubernetes.io/projected/8347e5af-bb13-45e0-9025-b43410d820b8-kube-api-access-gc9gk\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-etc-kubernetes\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-hosts-file\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-kubelet\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-ovnkube-config\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738184    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-log-socket\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738209    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-host\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738250    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l52ww\" (UniqueName: \"kubernetes.io/projected/bc6e6606-0356-41e9-b18d-a646137ee771-kube-api-access-l52ww\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.739108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-cnibin\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-sysctl-d\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/bc6e6606-0356-41e9-b18d-a646137ee771-socket-dir\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738522    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/eab09017-f4c7-4571-854c-ea23e76f8dfe-cnibin\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-cni-netd\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.738865    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c8395345-90a8-48e4-8fc1-aa9545d0a98a-cni-binary-copy\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.739137    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/8347e5af-bb13-45e0-9025-b43410d820b8-ovnkube-script-lib\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.739215    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-multus-certs\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.739847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.739778    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/eab09017-f4c7-4571-854c-ea23e76f8dfe-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.740177 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.739860    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.740177 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.739922    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-netns\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.740322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-log-socket\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.740430    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2514581-9d70-45d3-8cb2-3d6e0734d559-iptables-alerter-script\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.740511    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-host\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.740672    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-etc-kubernetes\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.740815    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.740899    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-hosts-file\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.740946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/8347e5af-bb13-45e0-9025-b43410d820b8-host-kubelet\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.741024    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-cnibin\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.741069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-run\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.741571    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c8395345-90a8-48e4-8fc1-aa9545d0a98a-multus-daemon-config\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.742311 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.741717    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret podName:4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:45.241692926 +0000 UTC m=+3.026707888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret") pod "global-pull-secret-syncer-h9wst" (UID: "4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:44.743674 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.743652    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c395bef-58b3-45ef-b677-db43aa258be1-etc-modprobe-d\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.743877 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.743858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c8395345-90a8-48e4-8fc1-aa9545d0a98a-host-run-k8s-cni-cncf-io\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.744153 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.744136    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:27:44.744284 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.744271    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:27:44.744377 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.744368    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ddbh2 for pod openshift-network-diagnostics/network-check-target-dztrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:44.744545 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.744530    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2 podName:09d49093-3630-48d2-b929-6e86d252faad nodeName:}" failed. No retries permitted until 2026-07-21 12:27:45.244508583 +0000 UTC m=+3.029523550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-ddbh2" (UniqueName: "kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2") pod "network-check-target-dztrl" (UID: "09d49093-3630-48d2-b929-6e86d252faad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:44.748937 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.748911    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/8347e5af-bb13-45e0-9025-b43410d820b8-ovn-node-metrics-cert\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.749593 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.749572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6trb\" (UniqueName: \"kubernetes.io/projected/c8395345-90a8-48e4-8fc1-aa9545d0a98a-kube-api-access-t6trb\") pod \"multus-qff6d\" (UID: \"c8395345-90a8-48e4-8fc1-aa9545d0a98a\") " pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.749832 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.749578    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqfhc\" (UniqueName: \"kubernetes.io/projected/7c395bef-58b3-45ef-b677-db43aa258be1-kube-api-access-qqfhc\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.752560 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.751687    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njwmp\" (UniqueName: \"kubernetes.io/projected/e2514581-9d70-45d3-8cb2-3d6e0734d559-kube-api-access-njwmp\") pod \"iptables-alerter-k4tt7\" (UID: \"e2514581-9d70-45d3-8cb2-3d6e0734d559\") " pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.754601 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.754554    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc9gk\" (UniqueName: \"kubernetes.io/projected/8347e5af-bb13-45e0-9025-b43410d820b8-kube-api-access-gc9gk\") pod \"ovnkube-node-lqpqc\" (UID: \"8347e5af-bb13-45e0-9025-b43410d820b8\") " pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.755207 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.754743    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c395bef-58b3-45ef-b677-db43aa258be1-tmp\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.755207 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.755155    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7wwp\" (UniqueName: \"kubernetes.io/projected/eab09017-f4c7-4571-854c-ea23e76f8dfe-kube-api-access-c7wwp\") pod \"multus-additional-cni-plugins-c5fnk\" (UID: \"eab09017-f4c7-4571-854c-ea23e76f8dfe\") " pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.755950 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.755895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsqxg\" (UniqueName: \"kubernetes.io/projected/5f1a2f32-365c-48b2-b9a8-56a5da0c42f0-kube-api-access-wsqxg\") pod \"node-resolver-76zlc\" (UID: \"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0\") " pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.757289 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.757245    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c395bef-58b3-45ef-b677-db43aa258be1-etc-tuned\") pod \"tuned-rplnb\" (UID: \"7c395bef-58b3-45ef-b677-db43aa258be1\") " pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.757879 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.757858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l52ww\" (UniqueName: \"kubernetes.io/projected/bc6e6606-0356-41e9-b18d-a646137ee771-kube-api-access-l52ww\") pod \"aws-ebs-csi-driver-node-db9hp\" (UID: \"bc6e6606-0356-41e9-b18d-a646137ee771\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.777080 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.776950    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:27:44.838596 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838566    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9f6ab4a5-3a99-439c-934c-e07efea84081-serviceca\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.838596 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/157a0b56-df6f-4a64-800a-84d8a7596eb9-konnectivity-ca\") pod \"konnectivity-agent-z5xzk\" (UID: \"157a0b56-df6f-4a64-800a-84d8a7596eb9\") " pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/157a0b56-df6f-4a64-800a-84d8a7596eb9-agent-certs\") pod \"konnectivity-agent-z5xzk\" (UID: \"157a0b56-df6f-4a64-800a-84d8a7596eb9\") " pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fhwr\" (UniqueName: \"kubernetes.io/projected/94d09ea1-66fe-4ee3-95ef-65866d92821c-kube-api-access-2fhwr\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.838746    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838762    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f6ab4a5-3a99-439c-934c-e07efea84081-host\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:44.838805    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs podName:94d09ea1-66fe-4ee3-95ef-65866d92821c nodeName:}" failed. No retries permitted until 2026-07-21 12:27:45.338785363 +0000 UTC m=+3.123800307 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs") pod "network-metrics-daemon-kmkdn" (UID: "94d09ea1-66fe-4ee3-95ef-65866d92821c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838822    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9f6ab4a5-3a99-439c-934c-e07efea84081-host\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.838843 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.838838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb6k8\" (UniqueName: \"kubernetes.io/projected/9f6ab4a5-3a99-439c-934c-e07efea84081-kube-api-access-qb6k8\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.839234 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.839078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9f6ab4a5-3a99-439c-934c-e07efea84081-serviceca\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.839356 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.839336    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/157a0b56-df6f-4a64-800a-84d8a7596eb9-konnectivity-ca\") pod \"konnectivity-agent-z5xzk\" (UID: \"157a0b56-df6f-4a64-800a-84d8a7596eb9\") " pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:44.841163 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.841143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/157a0b56-df6f-4a64-800a-84d8a7596eb9-agent-certs\") pod \"konnectivity-agent-z5xzk\" (UID: \"157a0b56-df6f-4a64-800a-84d8a7596eb9\") " pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:44.849344 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.849324    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fhwr\" (UniqueName: \"kubernetes.io/projected/94d09ea1-66fe-4ee3-95ef-65866d92821c-kube-api-access-2fhwr\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:44.849445 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.849326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb6k8\" (UniqueName: \"kubernetes.io/projected/9f6ab4a5-3a99-439c-934c-e07efea84081-kube-api-access-qb6k8\") pod \"node-ca-zsqm9\" (UID: \"9f6ab4a5-3a99-439c-934c-e07efea84081\") " pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.918363 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.918317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-76zlc"
Jul 21 12:27:44.931142 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.931114    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-c5fnk"
Jul 21 12:27:44.941924 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.941900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-qff6d"
Jul 21 12:27:44.946940 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.946917    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:27:44.955688 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.955661    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rplnb"
Jul 21 12:27:44.971481 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.971459    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 21 12:27:44.971656 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.971633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-k4tt7"
Jul 21 12:27:44.978355 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.978338    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp"
Jul 21 12:27:44.986996 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.986980    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-zsqm9"
Jul 21 12:27:44.992706 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:44.992687    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:27:45.242060 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.242032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:45.242221 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.242152    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:45.242221 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.242207    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret podName:4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:46.242194145 +0000 UTC m=+4.027209089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret") pod "global-pull-secret-syncer-h9wst" (UID: "4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:45.342688 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.342656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:45.342865 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.342698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:45.342865 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.342808    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:45.342865 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.342842    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:27:45.342865 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.342858    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:27:45.343061 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.342870    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ddbh2 for pod openshift-network-diagnostics/network-check-target-dztrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:45.343061 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.342878    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs podName:94d09ea1-66fe-4ee3-95ef-65866d92821c nodeName:}" failed. No retries permitted until 2026-07-21 12:27:46.342857565 +0000 UTC m=+4.127872518 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs") pod "network-metrics-daemon-kmkdn" (UID: "94d09ea1-66fe-4ee3-95ef-65866d92821c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:45.343061 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.342918    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2 podName:09d49093-3630-48d2-b929-6e86d252faad nodeName:}" failed. No retries permitted until 2026-07-21 12:27:46.342902478 +0000 UTC m=+4.127917442 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddbh2" (UniqueName: "kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2") pod "network-check-target-dztrl" (UID: "09d49093-3630-48d2-b929-6e86d252faad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:45.660050 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.659971    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:22:44 +0000 UTC" deadline="2028-03-08 18:25:10.203194755 +0000 UTC"
Jul 21 12:27:45.660050 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.660005    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14309h57m24.543192753s"
Jul 21 12:27:45.725606 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.725562    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc6e6606_0356_41e9_b18d_a646137ee771.slice/crio-d08bef7d8e7e94c5305d2e1fba79e0d79280839a82c3c3df3f3ae9510c07cd96 WatchSource:0}: Error finding container d08bef7d8e7e94c5305d2e1fba79e0d79280839a82c3c3df3f3ae9510c07cd96: Status 404 returned error can't find the container with id d08bef7d8e7e94c5305d2e1fba79e0d79280839a82c3c3df3f3ae9510c07cd96
Jul 21 12:27:45.744097 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.744066    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8395345_90a8_48e4_8fc1_aa9545d0a98a.slice/crio-6c3de1093c2f757568f29584cecda97470e023027f92a290f3d06b2eec79f9e5 WatchSource:0}: Error finding container 6c3de1093c2f757568f29584cecda97470e023027f92a290f3d06b2eec79f9e5: Status 404 returned error can't find the container with id 6c3de1093c2f757568f29584cecda97470e023027f92a290f3d06b2eec79f9e5
Jul 21 12:27:45.744818 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.744721    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c395bef_58b3_45ef_b677_db43aa258be1.slice/crio-c725e2cd178ec08e20a1e93c32ed5c209342f55d9eb7262de1332446fd0f73e2 WatchSource:0}: Error finding container c725e2cd178ec08e20a1e93c32ed5c209342f55d9eb7262de1332446fd0f73e2: Status 404 returned error can't find the container with id c725e2cd178ec08e20a1e93c32ed5c209342f55d9eb7262de1332446fd0f73e2
Jul 21 12:27:45.745752 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.745636    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8347e5af_bb13_45e0_9025_b43410d820b8.slice/crio-d35d9e00e41d1423818c905c4a151c1622b2dd3a60785fbc5b3fb3c226f13d3f WatchSource:0}: Error finding container d35d9e00e41d1423818c905c4a151c1622b2dd3a60785fbc5b3fb3c226f13d3f: Status 404 returned error can't find the container with id d35d9e00e41d1423818c905c4a151c1622b2dd3a60785fbc5b3fb3c226f13d3f
Jul 21 12:27:45.746518 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.746499    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f6ab4a5_3a99_439c_934c_e07efea84081.slice/crio-b728c9735f241573e053dc30bdd9611ddb20889b96b0d046627ea6b51eb5229d WatchSource:0}: Error finding container b728c9735f241573e053dc30bdd9611ddb20889b96b0d046627ea6b51eb5229d: Status 404 returned error can't find the container with id b728c9735f241573e053dc30bdd9611ddb20889b96b0d046627ea6b51eb5229d
Jul 21 12:27:45.747312 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.747241    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2514581_9d70_45d3_8cb2_3d6e0734d559.slice/crio-376670eb27d7f60f955bb2ceb6af7bcdf48fb108ffffbc75550b655b332aec59 WatchSource:0}: Error finding container 376670eb27d7f60f955bb2ceb6af7bcdf48fb108ffffbc75550b655b332aec59: Status 404 returned error can't find the container with id 376670eb27d7f60f955bb2ceb6af7bcdf48fb108ffffbc75550b655b332aec59
Jul 21 12:27:45.748318 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.748258    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157a0b56_df6f_4a64_800a_84d8a7596eb9.slice/crio-7b482b49ce8fa8c2c9800d745539ccecd5eeaa1dba163ade515ea902f87b01e0 WatchSource:0}: Error finding container 7b482b49ce8fa8c2c9800d745539ccecd5eeaa1dba163ade515ea902f87b01e0: Status 404 returned error can't find the container with id 7b482b49ce8fa8c2c9800d745539ccecd5eeaa1dba163ade515ea902f87b01e0
Jul 21 12:27:45.749855 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.749719    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeab09017_f4c7_4571_854c_ea23e76f8dfe.slice/crio-0afd7a292e52aafaf5c0c7a4dd240f3eada087f86fe021aacad778c8ef2e380a WatchSource:0}: Error finding container 0afd7a292e52aafaf5c0c7a4dd240f3eada087f86fe021aacad778c8ef2e380a: Status 404 returned error can't find the container with id 0afd7a292e52aafaf5c0c7a4dd240f3eada087f86fe021aacad778c8ef2e380a
Jul 21 12:27:45.750777 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:27:45.750703    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f1a2f32_365c_48b2_b9a8_56a5da0c42f0.slice/crio-affe310ab519cdea74d1aefecd11a7f9e0c25afc41e0dcdd99d4ea85e9b2b864 WatchSource:0}: Error finding container affe310ab519cdea74d1aefecd11a7f9e0c25afc41e0dcdd99d4ea85e9b2b864: Status 404 returned error can't find the container with id affe310ab519cdea74d1aefecd11a7f9e0c25afc41e0dcdd99d4ea85e9b2b864
Jul 21 12:27:45.826781 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.826750    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:45.826928 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:45.826894    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:45.835405 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.835367    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp" event={"ID":"bc6e6606-0356-41e9-b18d-a646137ee771","Type":"ContainerStarted","Data":"d08bef7d8e7e94c5305d2e1fba79e0d79280839a82c3c3df3f3ae9510c07cd96"}
Jul 21 12:27:45.837356 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.837325    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" event={"ID":"4823430b3b83d7b0cb59efe2028dc586","Type":"ContainerStarted","Data":"f9cf629e280cbb9263ee9258fa6c44ca3e517d0bff34c5c27a43ee103a14844f"}
Jul 21 12:27:45.838589 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.838566    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-76zlc" event={"ID":"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0","Type":"ContainerStarted","Data":"affe310ab519cdea74d1aefecd11a7f9e0c25afc41e0dcdd99d4ea85e9b2b864"}
Jul 21 12:27:45.839756 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.839736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerStarted","Data":"0afd7a292e52aafaf5c0c7a4dd240f3eada087f86fe021aacad778c8ef2e380a"}
Jul 21 12:27:45.841011 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.840984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z5xzk" event={"ID":"157a0b56-df6f-4a64-800a-84d8a7596eb9","Type":"ContainerStarted","Data":"7b482b49ce8fa8c2c9800d745539ccecd5eeaa1dba163ade515ea902f87b01e0"}
Jul 21 12:27:45.842074 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.842051    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4tt7" event={"ID":"e2514581-9d70-45d3-8cb2-3d6e0734d559","Type":"ContainerStarted","Data":"376670eb27d7f60f955bb2ceb6af7bcdf48fb108ffffbc75550b655b332aec59"}
Jul 21 12:27:45.843229 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.843200    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zsqm9" event={"ID":"9f6ab4a5-3a99-439c-934c-e07efea84081","Type":"ContainerStarted","Data":"b728c9735f241573e053dc30bdd9611ddb20889b96b0d046627ea6b51eb5229d"}
Jul 21 12:27:45.844362 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.844341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rplnb" event={"ID":"7c395bef-58b3-45ef-b677-db43aa258be1","Type":"ContainerStarted","Data":"c725e2cd178ec08e20a1e93c32ed5c209342f55d9eb7262de1332446fd0f73e2"}
Jul 21 12:27:45.845443 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.845418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qff6d" event={"ID":"c8395345-90a8-48e4-8fc1-aa9545d0a98a","Type":"ContainerStarted","Data":"6c3de1093c2f757568f29584cecda97470e023027f92a290f3d06b2eec79f9e5"}
Jul 21 12:27:45.846686 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:45.846665    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"d35d9e00e41d1423818c905c4a151c1622b2dd3a60785fbc5b3fb3c226f13d3f"}
Jul 21 12:27:46.248393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.248145    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:46.248604 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.248584    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:46.248680 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.248669    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret podName:4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:48.248648597 +0000 UTC m=+6.033663544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret") pod "global-pull-secret-syncer-h9wst" (UID: "4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:46.349433 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.349392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:46.349615 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.349444    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:46.349680 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.349623    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:27:46.349680 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.349644    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:27:46.349680 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.349657    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ddbh2 for pod openshift-network-diagnostics/network-check-target-dztrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:46.349829 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.349722    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2 podName:09d49093-3630-48d2-b929-6e86d252faad nodeName:}" failed. No retries permitted until 2026-07-21 12:27:48.349703599 +0000 UTC m=+6.134718559 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddbh2" (UniqueName: "kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2") pod "network-check-target-dztrl" (UID: "09d49093-3630-48d2-b929-6e86d252faad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:46.350312 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.350126    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:46.350312 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.350180    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs podName:94d09ea1-66fe-4ee3-95ef-65866d92821c nodeName:}" failed. No retries permitted until 2026-07-21 12:27:48.350163824 +0000 UTC m=+6.135178780 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs") pod "network-metrics-daemon-kmkdn" (UID: "94d09ea1-66fe-4ee3-95ef-65866d92821c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:46.660378 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.660263    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 12:22:44 +0000 UTC" deadline="2027-12-15 05:35:07.906130359 +0000 UTC"
Jul 21 12:27:46.660378 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.660332    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12281h7m21.245801935s"
Jul 21 12:27:46.828166 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.828123    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:46.828342 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.828182    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:46.828342 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.828323    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:46.828464 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:46.828438    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:46.870166 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.870061    2577 generic.go:356] "Generic (PLEG): container finished" podID="4835d46be4ce0f9c6f53c4a51eee7a04" containerID="7d1dbfab08e075094e1ca6a7ac3559b72e036c39bec755954884d0d5077d5e7d" exitCode=0
Jul 21 12:27:46.871121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.871097    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" event={"ID":"4835d46be4ce0f9c6f53c4a51eee7a04","Type":"ContainerDied","Data":"7d1dbfab08e075094e1ca6a7ac3559b72e036c39bec755954884d0d5077d5e7d"}
Jul 21 12:27:46.887691 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:46.887625    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-143-161.ec2.internal" podStartSLOduration=2.887603683 podStartE2EDuration="2.887603683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:45.850272786 +0000 UTC m=+3.635287755" watchObservedRunningTime="2026-07-21 12:27:46.887603683 +0000 UTC m=+4.672618651"
Jul 21 12:27:47.826572 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:47.826499    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:47.827155 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:47.826722    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:47.891823 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:47.891784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" event={"ID":"4835d46be4ce0f9c6f53c4a51eee7a04","Type":"ContainerStarted","Data":"a85ff1555dc437d5c57c1b9e5ec36311c10922aa0b77aa2dd85dcf06dfa66762"}
Jul 21 12:27:47.908729 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:47.908667    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-143-161.ec2.internal" podStartSLOduration=3.908644741 podStartE2EDuration="3.908644741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:27:47.90857824 +0000 UTC m=+5.693593209" watchObservedRunningTime="2026-07-21 12:27:47.908644741 +0000 UTC m=+5.693659701"
Jul 21 12:27:48.270074 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:48.270036    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:48.270249 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.270196    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:48.270306 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.270259    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret podName:4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6 nodeName:}" failed. No retries permitted until 2026-07-21 12:27:52.270241085 +0000 UTC m=+10.055256034 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret") pod "global-pull-secret-syncer-h9wst" (UID: "4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:48.371144 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:48.371107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:48.371315 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:48.371165    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:48.371315 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.371264    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:48.371440 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.371383    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:27:48.371440 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.371398    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs podName:94d09ea1-66fe-4ee3-95ef-65866d92821c nodeName:}" failed. No retries permitted until 2026-07-21 12:27:52.371314662 +0000 UTC m=+10.156329623 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs") pod "network-metrics-daemon-kmkdn" (UID: "94d09ea1-66fe-4ee3-95ef-65866d92821c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:48.371440 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.371412    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:27:48.371440 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.371426    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ddbh2 for pod openshift-network-diagnostics/network-check-target-dztrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:48.371637 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.371480    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2 podName:09d49093-3630-48d2-b929-6e86d252faad nodeName:}" failed. No retries permitted until 2026-07-21 12:27:52.371463534 +0000 UTC m=+10.156478479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddbh2" (UniqueName: "kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2") pod "network-check-target-dztrl" (UID: "09d49093-3630-48d2-b929-6e86d252faad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:48.832713 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:48.832676    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:48.833416 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.832794    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:48.833416 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:48.833186    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:48.833416 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:48.833279    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:49.826781 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:49.826746    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:49.827010 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:49.826948    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:50.827805 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:50.827119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:50.827805 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:50.827257    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:50.827805 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:50.827134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:50.827805 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:50.827717    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:51.827034 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:51.826977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:51.827240 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:51.827145    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:52.307318 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:52.307274    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:52.307798 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.307428    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:52.307798 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.307490    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret podName:4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:00.307471426 +0000 UTC m=+18.092486387 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret") pod "global-pull-secret-syncer-h9wst" (UID: "4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:52.408090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:52.408140    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.408330    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.408352    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.408364    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ddbh2 for pod openshift-network-diagnostics/network-check-target-dztrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.408419    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2 podName:09d49093-3630-48d2-b929-6e86d252faad nodeName:}" failed. No retries permitted until 2026-07-21 12:28:00.408401854 +0000 UTC m=+18.193416821 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddbh2" (UniqueName: "kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2") pod "network-check-target-dztrl" (UID: "09d49093-3630-48d2-b929-6e86d252faad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.408821    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:52.408897 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.408872    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs podName:94d09ea1-66fe-4ee3-95ef-65866d92821c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:00.408855113 +0000 UTC m=+18.193870058 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs") pod "network-metrics-daemon-kmkdn" (UID: "94d09ea1-66fe-4ee3-95ef-65866d92821c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:27:52.828170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:52.828130    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:52.828342 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.828242    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:52.828342 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:52.828273    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:52.828477 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:52.828380    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:53.826774 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:53.826738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:53.827250 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:53.826884    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:54.826731 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:54.826693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:54.826902 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:54.826817    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:54.826902 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:54.826890    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:54.827350 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:54.827029    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:55.826728 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:55.826693    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:55.826896 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:55.826816    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:56.829417 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:56.829384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:56.829776 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:56.829508    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:56.829776 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:56.829558    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:56.829776 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:56.829654    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:57.826244 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:57.826200    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:57.826417 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:57.826369    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:27:58.831030 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:58.830991    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:27:58.831485 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:58.831001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:27:58.831485 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:58.831114    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:27:58.831485 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:58.831224    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:27:59.826310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:27:59.826275    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:27:59.826501 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:27:59.826416    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:00.369053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:00.369013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:00.369564 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.369189    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:28:00.369564 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.369276    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret podName:4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:16.369252828 +0000 UTC m=+34.154267786 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret") pod "global-pull-secret-syncer-h9wst" (UID: "4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:28:00.470293 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:00.470257    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:00.470460 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:00.470305    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:00.470460 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.470423    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:28:00.470460 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.470442    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:28:00.470460 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.470455    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:28:00.470580 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.470465    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ddbh2 for pod openshift-network-diagnostics/network-check-target-dztrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:28:00.470580 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.470494    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs podName:94d09ea1-66fe-4ee3-95ef-65866d92821c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:16.47047586 +0000 UTC m=+34.255490821 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs") pod "network-metrics-daemon-kmkdn" (UID: "94d09ea1-66fe-4ee3-95ef-65866d92821c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:28:00.470580 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.470509    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2 podName:09d49093-3630-48d2-b929-6e86d252faad nodeName:}" failed. No retries permitted until 2026-07-21 12:28:16.470502799 +0000 UTC m=+34.255517747 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddbh2" (UniqueName: "kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2") pod "network-check-target-dztrl" (UID: "09d49093-3630-48d2-b929-6e86d252faad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:28:00.830150 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:00.830119    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:00.830373 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.830236    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:00.830373 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:00.830291    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:00.830459 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:00.830385    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:01.826329 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:01.826295    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:01.826775 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:01.826408    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:02.828076 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:02.828035    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:02.828525 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:02.828160    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:02.828525 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:02.828224    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:02.828525 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:02.828371    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:03.826577 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:03.826540    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:03.826775 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:03.826687    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:04.832389 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.832360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:04.832778 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.832360    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:04.832778 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:04.832476    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:04.832778 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:04.832518    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:04.934694 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.934650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-zsqm9" event={"ID":"9f6ab4a5-3a99-439c-934c-e07efea84081","Type":"ContainerStarted","Data":"7ef7d5653c90520ef688c6c66ae28de7809dd234b9a0141ef3d967de8faff8a3"}
Jul 21 12:28:04.936331 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.936306    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rplnb" event={"ID":"7c395bef-58b3-45ef-b677-db43aa258be1","Type":"ContainerStarted","Data":"ae287a6c81a601199ab49920b4d50d0d691d9fc6993364e20557f851c9a93d33"}
Jul 21 12:28:04.937916 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.937887    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-qff6d" event={"ID":"c8395345-90a8-48e4-8fc1-aa9545d0a98a","Type":"ContainerStarted","Data":"d02d28edb04b59fd354165eda11f4dd2d4ccecde9a8441b261fa96a032aaed15"}
Jul 21 12:28:04.939396 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.939374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp" event={"ID":"bc6e6606-0356-41e9-b18d-a646137ee771","Type":"ContainerStarted","Data":"4a941b0cab4898374772edd67d73242cd0e628eae878a984e32b400fe11ea8bd"}
Jul 21 12:28:04.940823 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.940794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-76zlc" event={"ID":"5f1a2f32-365c-48b2-b9a8-56a5da0c42f0","Type":"ContainerStarted","Data":"e8d2c6650f5b5025e3400de939713eb6fc8f530b3225bdfbfdf2011a632fa141"}
Jul 21 12:28:04.942286 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.942266    2577 generic.go:356] "Generic (PLEG): container finished" podID="eab09017-f4c7-4571-854c-ea23e76f8dfe" containerID="b6635720932ac0e8baa11e3af4afe27244fd2dff1006d378c2ecb5a87459f424" exitCode=0
Jul 21 12:28:04.942386 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.942331    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerDied","Data":"b6635720932ac0e8baa11e3af4afe27244fd2dff1006d378c2ecb5a87459f424"}
Jul 21 12:28:04.943818 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.943794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-z5xzk" event={"ID":"157a0b56-df6f-4a64-800a-84d8a7596eb9","Type":"ContainerStarted","Data":"fb333b71facbf4b63429a6dd23866c92681fee281d7b0be20d8b6958c220eb97"}
Jul 21 12:28:04.949437 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.949401    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-zsqm9" podStartSLOduration=4.135119295 podStartE2EDuration="22.949387461s" totalImagesPullingTime="18.814268166s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:04.948878484 +0000 UTC m=+22.733893472" watchObservedRunningTime="2026-07-21 12:28:04.949387461 +0000 UTC m=+22.734402427"
Jul 21 12:28:04.963617 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:04.963562    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-76zlc" podStartSLOduration=4.195616509 podStartE2EDuration="22.96354275s" totalImagesPullingTime="18.767926241s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:04.963509783 +0000 UTC m=+22.748524750" watchObservedRunningTime="2026-07-21 12:28:04.96354275 +0000 UTC m=+22.748557718"
Jul 21 12:28:05.001330 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:05.001279    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-qff6d" podStartSLOduration=4.146647518 podStartE2EDuration="23.001263992s" totalImagesPullingTime="18.854616474s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:05.000840969 +0000 UTC m=+22.785855961" watchObservedRunningTime="2026-07-21 12:28:05.001263992 +0000 UTC m=+22.786278958"
Jul 21 12:28:05.018284 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:05.018239    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rplnb" podStartSLOduration=4.201281357 podStartE2EDuration="23.018224759s" totalImagesPullingTime="18.816943402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:05.018121959 +0000 UTC m=+22.803136925" watchObservedRunningTime="2026-07-21 12:28:05.018224759 +0000 UTC m=+22.803239724"
Jul 21 12:28:05.827124 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:05.826874    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:05.827281 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:05.827248    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:05.948820 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:05.948786    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"a54d13b7181c323ac81e5f617fa150b353737ce1f8a5f05615f7ef7659283c57"}
Jul 21 12:28:05.949632 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:05.948880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"ce900d9b7d2095951b55d95ab8d3d40d307ea5a29f738840d1288f3295f77982"}
Jul 21 12:28:05.949632 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:05.948933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"1fb96ad9cc1cef8dc6829578f7c4552e49482267473a6f91076ea7073dacd3e1"}
Jul 21 12:28:06.210467 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.210419    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 21 12:28:06.688016 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.687893    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T12:28:06.210443752Z","UUID":"c05d842f-bc44-4de0-be97-d7ccf4129335","Handler":null,"Name":"","Endpoint":""}
Jul 21 12:28:06.689681 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.689660    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 21 12:28:06.689825 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.689702    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 21 12:28:06.827264 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.827223    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:06.827522 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:06.827357    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:06.827598 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.827537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:06.827652 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:06.827639    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:06.954009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.953908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-k4tt7" event={"ID":"e2514581-9d70-45d3-8cb2-3d6e0734d559","Type":"ContainerStarted","Data":"7abb1367c37212307ee21e862fc41dd65446c87008d2939d07b0cecd2104cc25"}
Jul 21 12:28:06.958451 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.958424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"ccc30c8c2a5a38154a29bd4e244dc5bdaa98894eed29e142cdee244a06afc6d1"}
Jul 21 12:28:06.958597 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.958488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"097e4b318c130717bdc774019f37fd35a21c86a94dc7307bb863736b1a1b1923"}
Jul 21 12:28:06.958597 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.958524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"c9db25837ec98133bce797e9b6dd5db7fdc0abbc77e950f28a0dd98b3ff2aea9"}
Jul 21 12:28:06.960742 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.960712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp" event={"ID":"bc6e6606-0356-41e9-b18d-a646137ee771","Type":"ContainerStarted","Data":"1633c55b0112d286cb1a07aa54d15b586f7fe7960d06f768dbec92f70cd0f7b3"}
Jul 21 12:28:06.968880 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.968830    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-z5xzk" podStartSLOduration=10.24371529 podStartE2EDuration="24.968815454s" totalImagesPullingTime="14.725100164s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:05.033328232 +0000 UTC m=+22.818343198" watchObservedRunningTime="2026-07-21 12:28:06.968815454 +0000 UTC m=+24.753830418"
Jul 21 12:28:06.969261 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:06.969226    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-k4tt7" podStartSLOduration=6.156515678 podStartE2EDuration="24.969218717s" totalImagesPullingTime="18.812703039s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:06.968321448 +0000 UTC m=+24.753336416" watchObservedRunningTime="2026-07-21 12:28:06.969218717 +0000 UTC m=+24.754233683"
Jul 21 12:28:07.826752 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:07.826715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:07.826991 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:07.826840    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:07.965396 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:07.965354    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp" event={"ID":"bc6e6606-0356-41e9-b18d-a646137ee771","Type":"ContainerStarted","Data":"1f35178b13e380c8a00b8b2117da6d158b1039f75e01d9e2d02c82167e990f62"}
Jul 21 12:28:07.981715 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:07.981666    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-db9hp" podStartSLOduration=4.640771739 podStartE2EDuration="25.981651064s" totalImagesPullingTime="21.340879325s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:07.980418471 +0000 UTC m=+25.765433439" watchObservedRunningTime="2026-07-21 12:28:07.981651064 +0000 UTC m=+25.766666029"
Jul 21 12:28:08.827220 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:08.827187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:08.827408 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:08.827188    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:08.827408 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:08.827316    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:08.827408 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:08.827397    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:09.050042 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:09.050005    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:28:09.050668 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:09.050651    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:28:09.826759 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:09.826684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:09.826901 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:09.826799    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:09.971932 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:09.971898    2577 generic.go:356] "Generic (PLEG): container finished" podID="eab09017-f4c7-4571-854c-ea23e76f8dfe" containerID="054f596bc4db2c9d1a57a2ca3361a55d0cf04a38a4563999a383c9e36914a0a5" exitCode=0
Jul 21 12:28:09.972121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:09.971985    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerDied","Data":"054f596bc4db2c9d1a57a2ca3361a55d0cf04a38a4563999a383c9e36914a0a5"}
Jul 21 12:28:09.975458 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:09.975415    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"280d841c34c66f62c45eb8220d05bd468933b26c4d772ed913fa67321cab528a"}
Jul 21 12:28:10.827075 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:10.827040    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:10.827075 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:10.827065    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:10.827677 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:10.827153    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:10.827677 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:10.827279    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:11.826554 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:11.826381    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:11.826732 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:11.826641    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:11.982121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:11.982089    2577 generic.go:356] "Generic (PLEG): container finished" podID="eab09017-f4c7-4571-854c-ea23e76f8dfe" containerID="bc793e2e1b75cc4e2e45c32acfefd9f017452c0d91523fa42fadbcb3eee3cf13" exitCode=0
Jul 21 12:28:11.982708 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:11.982171    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerDied","Data":"bc793e2e1b75cc4e2e45c32acfefd9f017452c0d91523fa42fadbcb3eee3cf13"}
Jul 21 12:28:11.986023 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:11.986001    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" event={"ID":"8347e5af-bb13-45e0-9025-b43410d820b8","Type":"ContainerStarted","Data":"84ca751e5a695f3121ed77705243e12a1e665b90130160ee2096a330e7308a1b"}
Jul 21 12:28:11.986277 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:11.986260    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:28:11.986356 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:11.986334    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:28:11.986417 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:11.986386    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:28:12.001978 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.001934    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:28:12.002089 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.002050    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:28:12.045350 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.045305    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc" podStartSLOduration=10.589164518 podStartE2EDuration="30.04529005s" totalImagesPullingTime="19.456125532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:12.044287607 +0000 UTC m=+29.829302573" watchObservedRunningTime="2026-07-21 12:28:12.04529005 +0000 UTC m=+29.830305016"
Jul 21 12:28:12.205143 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.205065    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:28:12.205278 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.205167    2577 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 21 12:28:12.205713 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.205689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-z5xzk"
Jul 21 12:28:12.827241 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.827205    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:12.827405 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:12.827323    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:12.827405 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:12.827369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:12.827511 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:12.827475    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:13.826377 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:13.826347    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:13.826755 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:13.826453    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:13.993793 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:13.993761    2577 generic.go:356] "Generic (PLEG): container finished" podID="eab09017-f4c7-4571-854c-ea23e76f8dfe" containerID="89915907f0ff2c6d13e403c4cd6e9613e812a28c08e36af3c421e98760fdbe2b" exitCode=0
Jul 21 12:28:13.993942 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:13.993853    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerDied","Data":"89915907f0ff2c6d13e403c4cd6e9613e812a28c08e36af3c421e98760fdbe2b"}
Jul 21 12:28:14.231166 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:14.230865    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9wst"]
Jul 21 12:28:14.231333 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:14.231232    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:14.231410 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:14.231343    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:14.231602 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:14.231570    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dztrl"]
Jul 21 12:28:14.231728 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:14.231658    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:14.231783 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:14.231737    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:14.232279 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:14.232258    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kmkdn"]
Jul 21 12:28:14.232377 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:14.232350    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:14.232473 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:14.232453    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:15.827184 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:15.827112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:15.827184 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:15.827145    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:15.827659 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:15.827233    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:15.827659 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:15.827265    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:15.827659 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:15.827347    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:15.827659 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:15.827412    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:16.391270 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:16.391232    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:16.391497 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.391397    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 21 12:28:16.391497 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.391467    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret podName:4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:48.391452655 +0000 UTC m=+66.176467604 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret") pod "global-pull-secret-syncer-h9wst" (UID: "4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6") : object "kube-system"/"original-pull-secret" not registered
Jul 21 12:28:16.492551 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:16.492516    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:16.492551 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:16.492570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:16.492844 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.492714    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:28:16.492844 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.492756    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 21 12:28:16.492844 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.492775    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 21 12:28:16.492844 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.492788    2577 projected.go:196] Error preparing data for projected volume kube-api-access-ddbh2 for pod openshift-network-diagnostics/network-check-target-dztrl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:28:16.492844 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.492793    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs podName:94d09ea1-66fe-4ee3-95ef-65866d92821c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:48.492772126 +0000 UTC m=+66.277787093 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs") pod "network-metrics-daemon-kmkdn" (UID: "94d09ea1-66fe-4ee3-95ef-65866d92821c") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 21 12:28:16.492844 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:16.492834    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2 podName:09d49093-3630-48d2-b929-6e86d252faad nodeName:}" failed. No retries permitted until 2026-07-21 12:28:48.492819367 +0000 UTC m=+66.277834312 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-ddbh2" (UniqueName: "kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2") pod "network-check-target-dztrl" (UID: "09d49093-3630-48d2-b929-6e86d252faad") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 21 12:28:17.827050 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:17.826999    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:17.827050 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:17.827036    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:17.827050 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:17.827036    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:17.827534 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:17.827142    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-kmkdn" podUID="94d09ea1-66fe-4ee3-95ef-65866d92821c"
Jul 21 12:28:17.827534 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:17.827219    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-h9wst" podUID="4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6"
Jul 21 12:28:17.827534 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:17.827286    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-dztrl" podUID="09d49093-3630-48d2-b929-6e86d252faad"
Jul 21 12:28:18.488743 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.488716    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-143-161.ec2.internal" event="NodeReady"
Jul 21 12:28:18.488911 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.488852    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 21 12:28:18.531552 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.531521    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-bvnqn"]
Jul 21 12:28:18.534882 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.534861    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dwpsm"]
Jul 21 12:28:18.535049 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.535024    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.536611 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.536589    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:18.537319 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.537297    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 21 12:28:18.537425 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.537305    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-r5psm\""
Jul 21 12:28:18.537490 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.537459    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 21 12:28:18.538856 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.538823    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 21 12:28:18.538856 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.538836    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 21 12:28:18.538856 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.538841    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 21 12:28:18.539368 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.539323    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6v6zb\""
Jul 21 12:28:18.544454 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.544434    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvnqn"]
Jul 21 12:28:18.546708 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.546687    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwpsm"]
Jul 21 12:28:18.710251 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.710208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjbv7\" (UniqueName: \"kubernetes.io/projected/02461782-afbd-4b5a-b082-09179a2f4471-kube-api-access-pjbv7\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.710448 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.710297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8cb2539-aa9d-44c0-b732-669e6f748287-cert\") pod \"ingress-canary-dwpsm\" (UID: \"a8cb2539-aa9d-44c0-b732-669e6f748287\") " pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:18.710448 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.710417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.710568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.710463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02461782-afbd-4b5a-b082-09179a2f4471-tmp-dir\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.710568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.710496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lq6c2\" (UniqueName: \"kubernetes.io/projected/a8cb2539-aa9d-44c0-b732-669e6f748287-kube-api-access-lq6c2\") pod \"ingress-canary-dwpsm\" (UID: \"a8cb2539-aa9d-44c0-b732-669e6f748287\") " pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:18.710568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.710529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/02461782-afbd-4b5a-b082-09179a2f4471-config-volume\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.811631 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.811548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/02461782-afbd-4b5a-b082-09179a2f4471-config-volume\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.811631 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.811624    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjbv7\" (UniqueName: \"kubernetes.io/projected/02461782-afbd-4b5a-b082-09179a2f4471-kube-api-access-pjbv7\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.811862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.811652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8cb2539-aa9d-44c0-b732-669e6f748287-cert\") pod \"ingress-canary-dwpsm\" (UID: \"a8cb2539-aa9d-44c0-b732-669e6f748287\") " pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:18.811862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.811701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.811862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.811735    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02461782-afbd-4b5a-b082-09179a2f4471-tmp-dir\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.811862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.811764    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lq6c2\" (UniqueName: \"kubernetes.io/projected/a8cb2539-aa9d-44c0-b732-669e6f748287-kube-api-access-lq6c2\") pod \"ingress-canary-dwpsm\" (UID: \"a8cb2539-aa9d-44c0-b732-669e6f748287\") " pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:18.812069 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:18.811878    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:28:18.812069 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:18.811950    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls podName:02461782-afbd-4b5a-b082-09179a2f4471 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:19.311929217 +0000 UTC m=+37.096944170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls") pod "dns-default-bvnqn" (UID: "02461782-afbd-4b5a-b082-09179a2f4471") : secret "dns-default-metrics-tls" not found
Jul 21 12:28:18.812069 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.812055    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02461782-afbd-4b5a-b082-09179a2f4471-tmp-dir\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.812225 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.812124    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/02461782-afbd-4b5a-b082-09179a2f4471-config-volume\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.816600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.816572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a8cb2539-aa9d-44c0-b732-669e6f748287-cert\") pod \"ingress-canary-dwpsm\" (UID: \"a8cb2539-aa9d-44c0-b732-669e6f748287\") " pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:18.820195 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.820169    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjbv7\" (UniqueName: \"kubernetes.io/projected/02461782-afbd-4b5a-b082-09179a2f4471-kube-api-access-pjbv7\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:18.820341 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.820305    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lq6c2\" (UniqueName: \"kubernetes.io/projected/a8cb2539-aa9d-44c0-b732-669e6f748287-kube-api-access-lq6c2\") pod \"ingress-canary-dwpsm\" (UID: \"a8cb2539-aa9d-44c0-b732-669e6f748287\") " pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:18.857103 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:18.857078    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dwpsm"
Jul 21 12:28:19.299175 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.299135    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-bqr6l"]
Jul 21 12:28:19.301566 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.301543    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.304189 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.304162    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 21 12:28:19.304492 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.304473    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 21 12:28:19.305072 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.305050    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c9cc7\""
Jul 21 12:28:19.305466 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.305418    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 21 12:28:19.305570 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.305463    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 21 12:28:19.315760 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.315737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.315884 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.315785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c158b7a-8e3b-483d-995f-59ce8614629c-data-volume\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.315884 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.315845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvn2l\" (UniqueName: \"kubernetes.io/projected/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-api-access-bvn2l\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.315884 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.315880    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:19.316124 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.315938    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.316124 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.315990    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:28:19.316124 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.316004    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c158b7a-8e3b-483d-995f-59ce8614629c-crio-socket\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.316124 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.316041    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls podName:02461782-afbd-4b5a-b082-09179a2f4471 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:20.316028838 +0000 UTC m=+38.101043781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls") pod "dns-default-bvnqn" (UID: "02461782-afbd-4b5a-b082-09179a2f4471") : secret "dns-default-metrics-tls" not found
Jul 21 12:28:19.317439 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.317420    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqr6l"]
Jul 21 12:28:19.339684 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.339654    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"]
Jul 21 12:28:19.341776 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.341752    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"
Jul 21 12:28:19.344590 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.344571    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 21 12:28:19.344590 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.344592    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-6ct79\""
Jul 21 12:28:19.344913 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.344895    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 21 12:28:19.355000 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.354976    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"]
Jul 21 12:28:19.416921 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.416887    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.417121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.416942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c158b7a-8e3b-483d-995f-59ce8614629c-data-volume\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.417121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.417009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bvn2l\" (UniqueName: \"kubernetes.io/projected/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-api-access-bvn2l\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.417121 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.417045    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:19.417121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.417055    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5qr2\" (UniqueName: \"kubernetes.io/projected/2cc15a8d-64f3-49bd-a9f2-74a8f210f260-kube-api-access-q5qr2\") pod \"migrator-7cc9c7ffd7-nh25g\" (UID: \"2cc15a8d-64f3-49bd-a9f2-74a8f210f260\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"
Jul 21 12:28:19.417121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.417101    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.417121 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.417117    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:19.917096261 +0000 UTC m=+37.702111218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:19.417409 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.417162    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:28:19.417409 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.417182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c158b7a-8e3b-483d-995f-59ce8614629c-crio-socket\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.417409 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.417223    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:19.91720579 +0000 UTC m=+37.702220758 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : configmap "kube-rbac-proxy" not found
Jul 21 12:28:19.417409 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.417307    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5c158b7a-8e3b-483d-995f-59ce8614629c-data-volume\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.417409 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.417399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5c158b7a-8e3b-483d-995f-59ce8614629c-crio-socket\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.426023 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.426002    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvn2l\" (UniqueName: \"kubernetes.io/projected/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-api-access-bvn2l\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.518587 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.518387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5qr2\" (UniqueName: \"kubernetes.io/projected/2cc15a8d-64f3-49bd-a9f2-74a8f210f260-kube-api-access-q5qr2\") pod \"migrator-7cc9c7ffd7-nh25g\" (UID: \"2cc15a8d-64f3-49bd-a9f2-74a8f210f260\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"
Jul 21 12:28:19.526078 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.526052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dwpsm"]
Jul 21 12:28:19.529001 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:19.528951    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8cb2539_aa9d_44c0_b732_669e6f748287.slice/crio-92d9855f1c47c4a35bbb69c137f4802cf3d15cc9c33e1af84d9957dfb7d057c0 WatchSource:0}: Error finding container 92d9855f1c47c4a35bbb69c137f4802cf3d15cc9c33e1af84d9957dfb7d057c0: Status 404 returned error can't find the container with id 92d9855f1c47c4a35bbb69c137f4802cf3d15cc9c33e1af84d9957dfb7d057c0
Jul 21 12:28:19.530044 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.530014    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5qr2\" (UniqueName: \"kubernetes.io/projected/2cc15a8d-64f3-49bd-a9f2-74a8f210f260-kube-api-access-q5qr2\") pod \"migrator-7cc9c7ffd7-nh25g\" (UID: \"2cc15a8d-64f3-49bd-a9f2-74a8f210f260\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"
Jul 21 12:28:19.652060 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.652030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"
Jul 21 12:28:19.794365 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.794337    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g"]
Jul 21 12:28:19.797562 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:19.797532    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc15a8d_64f3_49bd_a9f2_74a8f210f260.slice/crio-ddfd9a7766fe24bb32990e35e43b430aae9d00eeb8651617ddeedc418e7fb133 WatchSource:0}: Error finding container ddfd9a7766fe24bb32990e35e43b430aae9d00eeb8651617ddeedc418e7fb133: Status 404 returned error can't find the container with id ddfd9a7766fe24bb32990e35e43b430aae9d00eeb8651617ddeedc418e7fb133
Jul 21 12:28:19.826724 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.826696    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:19.826724 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.826710    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:19.826885 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.826803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:19.830272 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.830250    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mz2qd\""
Jul 21 12:28:19.830381 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.830250    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:28:19.830381 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.830310    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:28:19.830554 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.830540    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:28:19.830602 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.830542    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-288sp\""
Jul 21 12:28:19.830602 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.830597    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:28:19.921119 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.921079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.921633 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:19.921146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:19.921633 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.921202    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:28:19.921633 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.921266    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:20.921250069 +0000 UTC m=+38.706265013 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : configmap "kube-rbac-proxy" not found
Jul 21 12:28:19.921633 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.921291    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:19.921633 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:19.921351    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:20.921335989 +0000 UTC m=+38.706350937 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:20.010845 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:20.010803    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g" event={"ID":"2cc15a8d-64f3-49bd-a9f2-74a8f210f260","Type":"ContainerStarted","Data":"ddfd9a7766fe24bb32990e35e43b430aae9d00eeb8651617ddeedc418e7fb133"}
Jul 21 12:28:20.012141 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:20.012104    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwpsm" event={"ID":"a8cb2539-aa9d-44c0-b732-669e6f748287","Type":"ContainerStarted","Data":"92d9855f1c47c4a35bbb69c137f4802cf3d15cc9c33e1af84d9957dfb7d057c0"}
Jul 21 12:28:20.015410 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:20.015375    2577 generic.go:356] "Generic (PLEG): container finished" podID="eab09017-f4c7-4571-854c-ea23e76f8dfe" containerID="fe6929aa4ce36f43058a30b99466f62cb444f53f7fd50b90b7860d45cbd44937" exitCode=0
Jul 21 12:28:20.015527 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:20.015438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerDied","Data":"fe6929aa4ce36f43058a30b99466f62cb444f53f7fd50b90b7860d45cbd44937"}
Jul 21 12:28:20.325187 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:20.325076    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:20.325187 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:20.325176    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:28:20.325378 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:20.325249    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls podName:02461782-afbd-4b5a-b082-09179a2f4471 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:22.32523019 +0000 UTC m=+40.110245141 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls") pod "dns-default-bvnqn" (UID: "02461782-afbd-4b5a-b082-09179a2f4471") : secret "dns-default-metrics-tls" not found
Jul 21 12:28:20.930696 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:20.930658    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:20.931172 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:20.930768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:20.931172 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:20.930860    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 21 12:28:20.931172 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:20.930921    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:22.930903454 +0000 UTC m=+40.715918412 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : configmap "kube-rbac-proxy" not found
Jul 21 12:28:20.931350 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:20.931337    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:20.931404 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:20.931391    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:22.93137516 +0000 UTC m=+40.716390119 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:21.022220 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.021998    2577 generic.go:356] "Generic (PLEG): container finished" podID="eab09017-f4c7-4571-854c-ea23e76f8dfe" containerID="3926cdf950fe2161d9c4295fe47ae862b82dad6dc424064d18ab1341286401dc" exitCode=0
Jul 21 12:28:21.022220 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.022079    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerDied","Data":"3926cdf950fe2161d9c4295fe47ae862b82dad6dc424064d18ab1341286401dc"}
Jul 21 12:28:21.049395 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.049376    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-76zlc_5f1a2f32-365c-48b2-b9a8-56a5da0c42f0/dns-node-resolver/0.log"
Jul 21 12:28:21.397209 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.397175    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-pqldj"]
Jul 21 12:28:21.409336 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.409303    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-pqldj"]
Jul 21 12:28:21.409525 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.409417    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.411912 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.411882    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 21 12:28:21.411912 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.411903    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 21 12:28:21.412403 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.412127    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 21 12:28:21.412403 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.412177    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-77shd\""
Jul 21 12:28:21.412403 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.412305    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 21 12:28:21.433118 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.433083    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e001aa45-b7d3-4313-a384-7b17bfa28e89-signing-cabundle\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.433272 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.433147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e001aa45-b7d3-4313-a384-7b17bfa28e89-signing-key\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.433272 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.433239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q97f\" (UniqueName: \"kubernetes.io/projected/e001aa45-b7d3-4313-a384-7b17bfa28e89-kube-api-access-4q97f\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.534408 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.534369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e001aa45-b7d3-4313-a384-7b17bfa28e89-signing-cabundle\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.534595 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.534434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e001aa45-b7d3-4313-a384-7b17bfa28e89-signing-key\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.534595 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.534486    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q97f\" (UniqueName: \"kubernetes.io/projected/e001aa45-b7d3-4313-a384-7b17bfa28e89-kube-api-access-4q97f\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.535312 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.535288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/e001aa45-b7d3-4313-a384-7b17bfa28e89-signing-cabundle\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.538535 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.538516    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/e001aa45-b7d3-4313-a384-7b17bfa28e89-signing-key\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.544166 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.544143    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q97f\" (UniqueName: \"kubernetes.io/projected/e001aa45-b7d3-4313-a384-7b17bfa28e89-kube-api-access-4q97f\") pod \"service-ca-58598695f7-pqldj\" (UID: \"e001aa45-b7d3-4313-a384-7b17bfa28e89\") " pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.720487 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.720400    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-pqldj"
Jul 21 12:28:21.890615 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:21.890441    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-pqldj"]
Jul 21 12:28:21.955804 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:21.955762    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode001aa45_b7d3_4313_a384_7b17bfa28e89.slice/crio-e0e4e4b57318e37b0f442ef04723f13ddc4bcde15d26ba46ff8da69bbaf0b676 WatchSource:0}: Error finding container e0e4e4b57318e37b0f442ef04723f13ddc4bcde15d26ba46ff8da69bbaf0b676: Status 404 returned error can't find the container with id e0e4e4b57318e37b0f442ef04723f13ddc4bcde15d26ba46ff8da69bbaf0b676
Jul 21 12:28:22.027586 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.027557    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" event={"ID":"eab09017-f4c7-4571-854c-ea23e76f8dfe","Type":"ContainerStarted","Data":"ef9f0b81bd765a9b1f424662fad142e4f1410763a670202d269280ac937fefed"}
Jul 21 12:28:22.029168 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.029143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g" event={"ID":"2cc15a8d-64f3-49bd-a9f2-74a8f210f260","Type":"ContainerStarted","Data":"0d33deee2f0d92f5baa42651fc6ae599379a18023fb3d151ad03f86f6485fcbd"}
Jul 21 12:28:22.029281 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.029195    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g" event={"ID":"2cc15a8d-64f3-49bd-a9f2-74a8f210f260","Type":"ContainerStarted","Data":"e7d67f46b3fcde43f3437a8f455559c5da487e383917139e0bbb356eace05cd4"}
Jul 21 12:28:22.030243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.030223    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-pqldj" event={"ID":"e001aa45-b7d3-4313-a384-7b17bfa28e89","Type":"ContainerStarted","Data":"e0e4e4b57318e37b0f442ef04723f13ddc4bcde15d26ba46ff8da69bbaf0b676"}
Jul 21 12:28:22.077893 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.077826    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-nh25g" podStartSLOduration=1.907344873 podStartE2EDuration="3.077803951s" totalImagesPullingTime="1.170459078s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:22.076857169 +0000 UTC m=+39.861872134" watchObservedRunningTime="2026-07-21 12:28:22.077803951 +0000 UTC m=+39.862818922"
Jul 21 12:28:22.078357 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.078325    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-c5fnk" podStartSLOduration=9.118404305 podStartE2EDuration="40.078316169s" totalImagesPullingTime="30.959911864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:22.058482738 +0000 UTC m=+39.843497706" watchObservedRunningTime="2026-07-21 12:28:22.078316169 +0000 UTC m=+39.863331140"
Jul 21 12:28:22.341399 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.341359    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:22.341575 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:22.341472    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:28:22.341575 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:22.341523    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls podName:02461782-afbd-4b5a-b082-09179a2f4471 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:26.34150975 +0000 UTC m=+44.126524694 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls") pod "dns-default-bvnqn" (UID: "02461782-afbd-4b5a-b082-09179a2f4471") : secret "dns-default-metrics-tls" not found
Jul 21 12:28:22.448144 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.448119    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zsqm9_9f6ab4a5-3a99-439c-934c-e07efea84081/node-ca/0.log"
Jul 21 12:28:22.945763 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.945717    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:22.945927 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.945836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:22.945927 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:22.945884    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:22.946073 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:22.945974    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:26.945938964 +0000 UTC m=+44.730953911 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:22.946430 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:22.946411    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5c158b7a-8e3b-483d-995f-59ce8614629c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:23.034765 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:23.034728    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dwpsm" event={"ID":"a8cb2539-aa9d-44c0-b732-669e6f748287","Type":"ContainerStarted","Data":"2d175939aa855eb22c4c00fb167e575ab63b85b93c9b46a89d41b6cc513e7b5c"}
Jul 21 12:28:23.053151 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:23.053010    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dwpsm" podStartSLOduration=2.612427194 podStartE2EDuration="5.052995504s" totalImagesPullingTime="2.44056831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:23.052507002 +0000 UTC m=+40.837521959" watchObservedRunningTime="2026-07-21 12:28:23.052995504 +0000 UTC m=+40.838010470"
Jul 21 12:28:25.040432 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:25.040394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-pqldj" event={"ID":"e001aa45-b7d3-4313-a384-7b17bfa28e89","Type":"ContainerStarted","Data":"a873704bbd03314ee72c5e64de8f42eabf0696268167061a66acb2992e27381c"}
Jul 21 12:28:25.057664 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:25.057612    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-pqldj" podStartSLOduration=1.615437831 podStartE2EDuration="4.057596687s" totalImagesPullingTime="2.442158856s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:25.057302648 +0000 UTC m=+42.842317615" watchObservedRunningTime="2026-07-21 12:28:25.057596687 +0000 UTC m=+42.842611656"
Jul 21 12:28:26.372408 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:26.372368    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:26.372889 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:26.372545    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 21 12:28:26.372889 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:26.372618    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls podName:02461782-afbd-4b5a-b082-09179a2f4471 nodeName:}" failed. No retries permitted until 2026-07-21 12:28:34.372596498 +0000 UTC m=+52.157611448 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls") pod "dns-default-bvnqn" (UID: "02461782-afbd-4b5a-b082-09179a2f4471") : secret "dns-default-metrics-tls" not found
Jul 21 12:28:26.977138 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:26.977100    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:26.977302 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:26.977223    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:26.977302 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:28:26.977275    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls podName:5c158b7a-8e3b-483d-995f-59ce8614629c nodeName:}" failed. No retries permitted until 2026-07-21 12:28:34.977262155 +0000 UTC m=+52.762277099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-bqr6l" (UID: "5c158b7a-8e3b-483d-995f-59ce8614629c") : secret "insights-runtime-extractor-tls" not found
Jul 21 12:28:34.431074 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:34.431038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:34.434457 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:34.434432    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/02461782-afbd-4b5a-b082-09179a2f4471-metrics-tls\") pod \"dns-default-bvnqn\" (UID: \"02461782-afbd-4b5a-b082-09179a2f4471\") " pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:34.450336 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:34.450311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:34.593160 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:34.593127    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-bvnqn"]
Jul 21 12:28:34.595941 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:34.595913    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02461782_afbd_4b5a_b082_09179a2f4471.slice/crio-261128c5bdbd9f29b8b73ba8c6e6bfb5ecaece4bbed57ccff793ee351f21308c WatchSource:0}: Error finding container 261128c5bdbd9f29b8b73ba8c6e6bfb5ecaece4bbed57ccff793ee351f21308c: Status 404 returned error can't find the container with id 261128c5bdbd9f29b8b73ba8c6e6bfb5ecaece4bbed57ccff793ee351f21308c
Jul 21 12:28:35.035543 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:35.035501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:35.037742 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:35.037712    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5c158b7a-8e3b-483d-995f-59ce8614629c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-bqr6l\" (UID: \"5c158b7a-8e3b-483d-995f-59ce8614629c\") " pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:35.066745 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:35.066711    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvnqn" event={"ID":"02461782-afbd-4b5a-b082-09179a2f4471","Type":"ContainerStarted","Data":"261128c5bdbd9f29b8b73ba8c6e6bfb5ecaece4bbed57ccff793ee351f21308c"}
Jul 21 12:28:35.212605 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:35.212567    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-bqr6l"
Jul 21 12:28:35.357595 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:35.357565    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-bqr6l"]
Jul 21 12:28:35.360140 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:35.360113    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c158b7a_8e3b_483d_995f_59ce8614629c.slice/crio-164a582fbc224fba07cb477d5aaa7e87041e792593fac3126f7ce8fc9b052930 WatchSource:0}: Error finding container 164a582fbc224fba07cb477d5aaa7e87041e792593fac3126f7ce8fc9b052930: Status 404 returned error can't find the container with id 164a582fbc224fba07cb477d5aaa7e87041e792593fac3126f7ce8fc9b052930
Jul 21 12:28:36.075380 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:36.075261    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqr6l" event={"ID":"5c158b7a-8e3b-483d-995f-59ce8614629c","Type":"ContainerStarted","Data":"164a582fbc224fba07cb477d5aaa7e87041e792593fac3126f7ce8fc9b052930"}
Jul 21 12:28:38.084497 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:38.084456    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvnqn" event={"ID":"02461782-afbd-4b5a-b082-09179a2f4471","Type":"ContainerStarted","Data":"b2b920d29ec1af8625302909b89f28a101bf428f57e0ec508a9dd9bef6c8223c"}
Jul 21 12:28:38.084998 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:38.084527    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-bvnqn" event={"ID":"02461782-afbd-4b5a-b082-09179a2f4471","Type":"ContainerStarted","Data":"9203a4a822ad6bb90fcc3c7c530797e168a8713ac1760483aedb0ce7f96956f2"}
Jul 21 12:28:38.084998 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:38.084664    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:38.106867 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:38.106816    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-bvnqn" podStartSLOduration=17.481291702 podStartE2EDuration="20.106798657s" totalImagesPullingTime="2.625506955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:38.106559494 +0000 UTC m=+55.891574472" watchObservedRunningTime="2026-07-21 12:28:38.106798657 +0000 UTC m=+55.891813627"
Jul 21 12:28:41.096508 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:41.096471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqr6l" event={"ID":"5c158b7a-8e3b-483d-995f-59ce8614629c","Type":"ContainerStarted","Data":"b5e2a99647002623fa3853caeeede8cdb4a972e1cfbbdff0896cd3a398cefce8"}
Jul 21 12:28:44.004640 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.004608    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lqpqc"
Jul 21 12:28:44.113944 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.113919    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"]
Jul 21 12:28:44.116459 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.116424    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"]
Jul 21 12:28:44.116597 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.116574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.118760 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.118744    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.133435 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.133417    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-qkbpl\""
Jul 21 12:28:44.133568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.133549    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 21 12:28:44.136710 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.136694    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 21 12:28:44.137184 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.137165    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 21 12:28:44.137402 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.137386    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 21 12:28:44.141581 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.141564    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-vsmn5\""
Jul 21 12:28:44.143568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.143554    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 21 12:28:44.144293 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.144280    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 21 12:28:44.156639 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.156616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"]
Jul 21 12:28:44.166247 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.166222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"]
Jul 21 12:28:44.208029 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.208002    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n9dz\" (UniqueName: \"kubernetes.io/projected/17a4bd4b-d401-4125-a6a0-d663ae9c359e-kube-api-access-5n9dz\") pod \"managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl\" (UID: \"17a4bd4b-d401-4125-a6a0-d663ae9c359e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.208150 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.208038    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17a4bd4b-d401-4125-a6a0-d663ae9c359e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl\" (UID: \"17a4bd4b-d401-4125-a6a0-d663ae9c359e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.208150 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.208069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e807dafc-b707-40f5-aaf7-e125c273bf7b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rg7w2\" (UID: \"e807dafc-b707-40f5-aaf7-e125c273bf7b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.208306 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.208237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e807dafc-b707-40f5-aaf7-e125c273bf7b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rg7w2\" (UID: \"e807dafc-b707-40f5-aaf7-e125c273bf7b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.210877 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.210854    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"]
Jul 21 12:28:44.213424 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.213408    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.215842 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.215822    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 21 12:28:44.216081 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.216066    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 21 12:28:44.216173 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.216086    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 21 12:28:44.220365 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.220347    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 21 12:28:44.257515 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.257464    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"]
Jul 21 12:28:44.309025 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.308998    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-hub\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.309135 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309051    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e807dafc-b707-40f5-aaf7-e125c273bf7b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rg7w2\" (UID: \"e807dafc-b707-40f5-aaf7-e125c273bf7b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.309135 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309101    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2szdb\" (UniqueName: \"kubernetes.io/projected/72599348-b487-4262-a263-7ee5d4023aac-kube-api-access-2szdb\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.309217 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309136    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n9dz\" (UniqueName: \"kubernetes.io/projected/17a4bd4b-d401-4125-a6a0-d663ae9c359e-kube-api-access-5n9dz\") pod \"managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl\" (UID: \"17a4bd4b-d401-4125-a6a0-d663ae9c359e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.309217 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309162    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72599348-b487-4262-a263-7ee5d4023aac-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.309217 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309190    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17a4bd4b-d401-4125-a6a0-d663ae9c359e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl\" (UID: \"17a4bd4b-d401-4125-a6a0-d663ae9c359e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.309339 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309216    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-ca\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.309339 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309239    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.309430 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e807dafc-b707-40f5-aaf7-e125c273bf7b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rg7w2\" (UID: \"e807dafc-b707-40f5-aaf7-e125c273bf7b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.309483 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.309932 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.309910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e807dafc-b707-40f5-aaf7-e125c273bf7b-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-rg7w2\" (UID: \"e807dafc-b707-40f5-aaf7-e125c273bf7b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.311467 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.311449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/17a4bd4b-d401-4125-a6a0-d663ae9c359e-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl\" (UID: \"17a4bd4b-d401-4125-a6a0-d663ae9c359e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.311548 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.311475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e807dafc-b707-40f5-aaf7-e125c273bf7b-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-rg7w2\" (UID: \"e807dafc-b707-40f5-aaf7-e125c273bf7b\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.339240 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.339216    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n9dz\" (UniqueName: \"kubernetes.io/projected/17a4bd4b-d401-4125-a6a0-d663ae9c359e-kube-api-access-5n9dz\") pod \"managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl\" (UID: \"17a4bd4b-d401-4125-a6a0-d663ae9c359e\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.410139 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.410113    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2szdb\" (UniqueName: \"kubernetes.io/projected/72599348-b487-4262-a263-7ee5d4023aac-kube-api-access-2szdb\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.410274 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.410143    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72599348-b487-4262-a263-7ee5d4023aac-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.410274 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.410163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-ca\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.410274 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.410179    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.410274 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.410199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.410485 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.410287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-hub\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.410936 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.410912    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/72599348-b487-4262-a263-7ee5d4023aac-additional-service-ca\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.412678 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.412658    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.412746 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.412683    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-hub\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.412899 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.412883    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.412899 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.412891    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/72599348-b487-4262-a263-7ee5d4023aac-ca\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.423469 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.423444    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2szdb\" (UniqueName: \"kubernetes.io/projected/72599348-b487-4262-a263-7ee5d4023aac-kube-api-access-2szdb\") pod \"cluster-proxy-proxy-agent-7cff668979-x6ll7\" (UID: \"72599348-b487-4262-a263-7ee5d4023aac\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.458308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.458289    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"
Jul 21 12:28:44.468934 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.468916    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"
Jul 21 12:28:44.522265 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.522215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"
Jul 21 12:28:44.591948 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.591914    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2"]
Jul 21 12:28:44.596125 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:44.596085    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode807dafc_b707_40f5_aaf7_e125c273bf7b.slice/crio-4efeb2dc50556b0ed03fb17c50e063bd27cf0f18ca8ace0650538dd37c0ab37f WatchSource:0}: Error finding container 4efeb2dc50556b0ed03fb17c50e063bd27cf0f18ca8ace0650538dd37c0ab37f: Status 404 returned error can't find the container with id 4efeb2dc50556b0ed03fb17c50e063bd27cf0f18ca8ace0650538dd37c0ab37f
Jul 21 12:28:44.620108 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.620078    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl"]
Jul 21 12:28:44.623176 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:44.623148    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17a4bd4b_d401_4125_a6a0_d663ae9c359e.slice/crio-9023be8fe67ac9c259ba520d2c78d6833dd9bc80d9bda642008247ce8a48673f WatchSource:0}: Error finding container 9023be8fe67ac9c259ba520d2c78d6833dd9bc80d9bda642008247ce8a48673f: Status 404 returned error can't find the container with id 9023be8fe67ac9c259ba520d2c78d6833dd9bc80d9bda642008247ce8a48673f
Jul 21 12:28:44.660338 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:44.660308    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7"]
Jul 21 12:28:44.664027 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:44.664003    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72599348_b487_4262_a263_7ee5d4023aac.slice/crio-fa8a0bc8e2aa278ceaa058c7a032d9600c7bce9472748882456962b033edee60 WatchSource:0}: Error finding container fa8a0bc8e2aa278ceaa058c7a032d9600c7bce9472748882456962b033edee60: Status 404 returned error can't find the container with id fa8a0bc8e2aa278ceaa058c7a032d9600c7bce9472748882456962b033edee60
Jul 21 12:28:45.110074 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:45.110038    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqr6l" event={"ID":"5c158b7a-8e3b-483d-995f-59ce8614629c","Type":"ContainerStarted","Data":"c1d1c4717713bc063e9523018cd50aea1e96d838952a9e0e635b83fa5b8ea7c3"}
Jul 21 12:28:45.111173 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:45.111148    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl" event={"ID":"17a4bd4b-d401-4125-a6a0-d663ae9c359e","Type":"ContainerStarted","Data":"9023be8fe67ac9c259ba520d2c78d6833dd9bc80d9bda642008247ce8a48673f"}
Jul 21 12:28:45.112170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:45.112150    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2" event={"ID":"e807dafc-b707-40f5-aaf7-e125c273bf7b","Type":"ContainerStarted","Data":"4efeb2dc50556b0ed03fb17c50e063bd27cf0f18ca8ace0650538dd37c0ab37f"}
Jul 21 12:28:45.113179 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:45.113157    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7" event={"ID":"72599348-b487-4262-a263-7ee5d4023aac","Type":"ContainerStarted","Data":"fa8a0bc8e2aa278ceaa058c7a032d9600c7bce9472748882456962b033edee60"}
Jul 21 12:28:48.091419 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.091383    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-bvnqn"
Jul 21 12:28:48.446342 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.446252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:48.448701 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.448677    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 21 12:28:48.458587 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.458567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6-original-pull-secret\") pod \"global-pull-secret-syncer-h9wst\" (UID: \"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6\") " pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:48.546739 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.546707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:48.546900 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.546755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:48.549470 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.549448    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 21 12:28:48.549590 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.549549    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 21 12:28:48.559779 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.559753    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 21 12:28:48.560206 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.560183    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/94d09ea1-66fe-4ee3-95ef-65866d92821c-metrics-certs\") pod \"network-metrics-daemon-kmkdn\" (UID: \"94d09ea1-66fe-4ee3-95ef-65866d92821c\") " pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:48.571081 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.571058    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbh2\" (UniqueName: \"kubernetes.io/projected/09d49093-3630-48d2-b929-6e86d252faad-kube-api-access-ddbh2\") pod \"network-check-target-dztrl\" (UID: \"09d49093-3630-48d2-b929-6e86d252faad\") " pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:48.639474 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.639445    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-288sp\""
Jul 21 12:28:48.644368 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.644337    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mz2qd\""
Jul 21 12:28:48.647180 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.647154    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-kmkdn"
Jul 21 12:28:48.647298 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.647215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-h9wst"
Jul 21 12:28:48.652376 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:48.652356    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:50.718994 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:50.718652    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-h9wst"]
Jul 21 12:28:50.724152 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:50.724104    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0a4fdc_9ec4_4894_96f7_2e68c62f4bc6.slice/crio-ae2f8007ed4a95e00f703644dc5a4467a8ff102fd293442656230395b1a6f38c WatchSource:0}: Error finding container ae2f8007ed4a95e00f703644dc5a4467a8ff102fd293442656230395b1a6f38c: Status 404 returned error can't find the container with id ae2f8007ed4a95e00f703644dc5a4467a8ff102fd293442656230395b1a6f38c
Jul 21 12:28:50.941435 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:50.941394    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-kmkdn"]
Jul 21 12:28:50.944410 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:50.944378    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d09ea1_66fe_4ee3_95ef_65866d92821c.slice/crio-8e2506514f6bb9446a5e599bab3ab4313adbb0b2c2c51a4aa88f4eba59900bad WatchSource:0}: Error finding container 8e2506514f6bb9446a5e599bab3ab4313adbb0b2c2c51a4aa88f4eba59900bad: Status 404 returned error can't find the container with id 8e2506514f6bb9446a5e599bab3ab4313adbb0b2c2c51a4aa88f4eba59900bad
Jul 21 12:28:50.945904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:50.945881    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-dztrl"]
Jul 21 12:28:50.949600 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:28:50.949578    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09d49093_3630_48d2_b929_6e86d252faad.slice/crio-f66a543b72b068142f82dfc03168254868e342cd030c43e817e586209a4b1b5f WatchSource:0}: Error finding container f66a543b72b068142f82dfc03168254868e342cd030c43e817e586209a4b1b5f: Status 404 returned error can't find the container with id f66a543b72b068142f82dfc03168254868e342cd030c43e817e586209a4b1b5f
Jul 21 12:28:51.135153 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.135057    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl" event={"ID":"17a4bd4b-d401-4125-a6a0-d663ae9c359e","Type":"ContainerStarted","Data":"b20cc03198e7dd7deb7835675b38e83585c65ad2fe41b6403ed938b993bf5ad0"}
Jul 21 12:28:51.136518 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.136486    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2" event={"ID":"e807dafc-b707-40f5-aaf7-e125c273bf7b","Type":"ContainerStarted","Data":"15c9190493c78faa545fe92506cbceadf2bfb8cc7a1484e250938d2a0d3bf431"}
Jul 21 12:28:51.138402 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.138370    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7" event={"ID":"72599348-b487-4262-a263-7ee5d4023aac","Type":"ContainerStarted","Data":"6b9d8a0098637e9603726e71117f690f70dfe0af47d26ec24c2370634082faec"}
Jul 21 12:28:51.138515 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.138408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7" event={"ID":"72599348-b487-4262-a263-7ee5d4023aac","Type":"ContainerStarted","Data":"dd7ac54fdaa18f7ac42865f370462d204d97ffa42d3d95a06bb12e94a9355c8f"}
Jul 21 12:28:51.138515 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.138444    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7" event={"ID":"72599348-b487-4262-a263-7ee5d4023aac","Type":"ContainerStarted","Data":"b81b942e23d4e1ae9bb466af8be1883e6cbe6e78570c43353ac13d785e548c68"}
Jul 21 12:28:51.139590 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.139568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dztrl" event={"ID":"09d49093-3630-48d2-b929-6e86d252faad","Type":"ContainerStarted","Data":"f66a543b72b068142f82dfc03168254868e342cd030c43e817e586209a4b1b5f"}
Jul 21 12:28:51.140634 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.140613    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmkdn" event={"ID":"94d09ea1-66fe-4ee3-95ef-65866d92821c","Type":"ContainerStarted","Data":"8e2506514f6bb9446a5e599bab3ab4313adbb0b2c2c51a4aa88f4eba59900bad"}
Jul 21 12:28:51.141655 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.141637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h9wst" event={"ID":"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6","Type":"ContainerStarted","Data":"ae2f8007ed4a95e00f703644dc5a4467a8ff102fd293442656230395b1a6f38c"}
Jul 21 12:28:51.143553 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.143534    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-bqr6l" event={"ID":"5c158b7a-8e3b-483d-995f-59ce8614629c","Type":"ContainerStarted","Data":"e4646f757839184c67635681dc77ff09b2eab602d90b8cb9971f4a0c49ca1cc4"}
Jul 21 12:28:51.149858 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.149811    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6c7b94b5fd-b9rbl" podStartSLOduration=1.238601326 podStartE2EDuration="7.149796788s" totalImagesPullingTime="5.911195462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:51.149702957 +0000 UTC m=+68.934717924" watchObservedRunningTime="2026-07-21 12:28:51.149796788 +0000 UTC m=+68.934811839"
Jul 21 12:28:51.165242 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.165203    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-rg7w2" podStartSLOduration=1.236330428 podStartE2EDuration="7.165191175s" totalImagesPullingTime="5.928860747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:51.164448034 +0000 UTC m=+68.949463025" watchObservedRunningTime="2026-07-21 12:28:51.165191175 +0000 UTC m=+68.950206141"
Jul 21 12:28:51.184053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.184010    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-7cff668979-x6ll7" podStartSLOduration=1.291626023 podStartE2EDuration="7.183996162s" totalImagesPullingTime="5.892370139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:28:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:51.18286362 +0000 UTC m=+68.967878587" watchObservedRunningTime="2026-07-21 12:28:51.183996162 +0000 UTC m=+68.969011127"
Jul 21 12:28:51.201300 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:51.201252    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-bqr6l" podStartSLOduration=17.15177583 podStartE2EDuration="32.201237351s" totalImagesPullingTime="15.049461521s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:28:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:51.200122053 +0000 UTC m=+68.985137017" watchObservedRunningTime="2026-07-21 12:28:51.201237351 +0000 UTC m=+68.986252318"
Jul 21 12:28:53.153588 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:53.153498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmkdn" event={"ID":"94d09ea1-66fe-4ee3-95ef-65866d92821c","Type":"ContainerStarted","Data":"9cc0df3c93ededff0dc2d800f6003b0fe69006b0a4a06eb3116a430fa20f1f3a"}
Jul 21 12:28:53.153588 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:53.153572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-kmkdn" event={"ID":"94d09ea1-66fe-4ee3-95ef-65866d92821c","Type":"ContainerStarted","Data":"b5e511235ac428329cb595e1997125f3666d1e34899b23d94fa7c2b26c5c1622"}
Jul 21 12:28:53.170282 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:53.170127    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-kmkdn" podStartSLOduration=69.642888269 podStartE2EDuration="1m11.170107554s" totalImagesPullingTime="1.527219285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:53.16910276 +0000 UTC m=+70.954117749" watchObservedRunningTime="2026-07-21 12:28:53.170107554 +0000 UTC m=+70.955122520"
Jul 21 12:28:57.168902 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:57.168866    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-h9wst" event={"ID":"4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6","Type":"ContainerStarted","Data":"a9da013308d7d3e738a5f47e93b21faa63f4e59586039a1d30e90c178875b602"}
Jul 21 12:28:57.170277 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:57.170253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-dztrl" event={"ID":"09d49093-3630-48d2-b929-6e86d252faad","Type":"ContainerStarted","Data":"3cb546c42fc1289ee366a85ead2a69fdd9390755e077570df3f7daed226e5af8"}
Jul 21 12:28:57.170385 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:57.170377    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:28:57.191585 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:57.191537    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-h9wst" podStartSLOduration=69.730561501 podStartE2EDuration="1m15.191524451s" totalImagesPullingTime="5.46096295s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:57.190599699 +0000 UTC m=+74.975614659" watchObservedRunningTime="2026-07-21 12:28:57.191524451 +0000 UTC m=+74.976539450"
Jul 21 12:28:57.206920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:28:57.206876    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-dztrl" podStartSLOduration=69.971760887 podStartE2EDuration="1m15.206861143s" totalImagesPullingTime="5.235100256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:27:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:28:57.206401577 +0000 UTC m=+74.991416570" watchObservedRunningTime="2026-07-21 12:28:57.206861143 +0000 UTC m=+74.991876110"
Jul 21 12:29:11.764397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.764269    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8j7h4"]
Jul 21 12:29:11.863767 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.863738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.866409 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.866386    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 21 12:29:11.866650 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.866433    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 21 12:29:11.866736 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.866454    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 21 12:29:11.867523 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.867240    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 21 12:29:11.867523 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.867247    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 21 12:29:11.867523 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.867370    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sjw6b\""
Jul 21 12:29:11.867523 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.867481    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 21 12:29:11.905166 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw5hl\" (UniqueName: \"kubernetes.io/projected/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-kube-api-access-tw5hl\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905323 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-sys\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905323 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-accelerators-collector-config\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905323 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-textfile\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905323 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-root\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905475 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905475 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905397    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-tls\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905475 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905422    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-metrics-client-ca\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:11.905475 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:11.905448    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-wtmp\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.006609 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.006862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006625    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-tls\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.006862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006688    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-metrics-client-ca\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.006862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006731    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-wtmp\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.006862 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:29:12.006738    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 21 12:29:12.006862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5hl\" (UniqueName: \"kubernetes.io/projected/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-kube-api-access-tw5hl\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.006862 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:29:12.006807    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-tls podName:08f3b63c-95ca-45c5-81e9-b4d7c12e1451 nodeName:}" failed. No retries permitted until 2026-07-21 12:29:12.506785763 +0000 UTC m=+90.291800714 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-tls") pod "node-exporter-8j7h4" (UID: "08f3b63c-95ca-45c5-81e9-b4d7c12e1451") : secret "node-exporter-tls" not found
Jul 21 12:29:12.006862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006834    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-sys\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007280 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-accelerators-collector-config\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007280 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-textfile\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007280 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.006938    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-root\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007280 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.007073    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-root\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007280 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.007247    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-wtmp\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007526 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.007282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-sys\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007526 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.007364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-metrics-client-ca\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007631 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.007593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-textfile\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.007805 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.007779    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-accelerators-collector-config\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.009555 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.009528    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.016269 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.016197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5hl\" (UniqueName: \"kubernetes.io/projected/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-kube-api-access-tw5hl\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.510974 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.510921    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-tls\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.513219 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.513195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/08f3b63c-95ca-45c5-81e9-b4d7c12e1451-node-exporter-tls\") pod \"node-exporter-8j7h4\" (UID: \"08f3b63c-95ca-45c5-81e9-b4d7c12e1451\") " pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.774862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:12.774779    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8j7h4"
Jul 21 12:29:12.833858 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:29:12.833832    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08f3b63c_95ca_45c5_81e9_b4d7c12e1451.slice/crio-cd315c42771ab7a341a40ed9cbdec9609c4cf3436a5deb738bdb3735f0ee699f WatchSource:0}: Error finding container cd315c42771ab7a341a40ed9cbdec9609c4cf3436a5deb738bdb3735f0ee699f: Status 404 returned error can't find the container with id cd315c42771ab7a341a40ed9cbdec9609c4cf3436a5deb738bdb3735f0ee699f
Jul 21 12:29:13.226100 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:13.226010    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8j7h4" event={"ID":"08f3b63c-95ca-45c5-81e9-b4d7c12e1451","Type":"ContainerStarted","Data":"cd315c42771ab7a341a40ed9cbdec9609c4cf3436a5deb738bdb3735f0ee699f"}
Jul 21 12:29:14.230442 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:14.230403    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8j7h4" event={"ID":"08f3b63c-95ca-45c5-81e9-b4d7c12e1451","Type":"ContainerStarted","Data":"836898e700659bbb0b59207abbf416acd46cc21d533bceedd51545728a12b5b0"}
Jul 21 12:29:15.235529 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:15.235494    2577 generic.go:356] "Generic (PLEG): container finished" podID="08f3b63c-95ca-45c5-81e9-b4d7c12e1451" containerID="836898e700659bbb0b59207abbf416acd46cc21d533bceedd51545728a12b5b0" exitCode=0
Jul 21 12:29:15.235975 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:15.235584    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8j7h4" event={"ID":"08f3b63c-95ca-45c5-81e9-b4d7c12e1451","Type":"ContainerDied","Data":"836898e700659bbb0b59207abbf416acd46cc21d533bceedd51545728a12b5b0"}
Jul 21 12:29:16.241379 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:16.241343    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8j7h4" event={"ID":"08f3b63c-95ca-45c5-81e9-b4d7c12e1451","Type":"ContainerStarted","Data":"263f9090a5c597acc6b724d30a29eae0525bf19621b22135efc64f6a1bf87854"}
Jul 21 12:29:16.241751 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:16.241404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8j7h4" event={"ID":"08f3b63c-95ca-45c5-81e9-b4d7c12e1451","Type":"ContainerStarted","Data":"51e26327de0df163796ff18f88ca7ae616b0a0a05969298d41e2664dec4c4fae"}
Jul 21 12:29:16.261119 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:16.261070    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8j7h4" podStartSLOduration=3.884086441 podStartE2EDuration="5.261054361s" totalImagesPullingTime="1.288641528s" totalInitContainerRuntime="88.326392ms" isStatefulPod=true podCreationTimestamp="2026-07-21 12:29:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:29:16.25970377 +0000 UTC m=+94.044718742" watchObservedRunningTime="2026-07-21 12:29:16.261054361 +0000 UTC m=+94.046069327"
Jul 21 12:29:28.176569 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:29:28.176533    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-dztrl"
Jul 21 12:31:58.648260 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.648230    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j777b"]
Jul 21 12:31:58.650534 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.650517    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:58.653700 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.653681    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-crn75\""
Jul 21 12:31:58.654144 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.654127    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 21 12:31:58.654518 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.654506    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 21 12:31:58.663399 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.663381    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j777b"]
Jul 21 12:31:58.740446 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.740420    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dzqk\" (UniqueName: \"kubernetes.io/projected/4c2572c2-6ab4-4e15-9497-8f8cb707c012-kube-api-access-6dzqk\") pod \"cert-manager-cainjector-559cdd8c87-j777b\" (UID: \"4c2572c2-6ab4-4e15-9497-8f8cb707c012\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:58.740561 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.740462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c2572c2-6ab4-4e15-9497-8f8cb707c012-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j777b\" (UID: \"4c2572c2-6ab4-4e15-9497-8f8cb707c012\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:58.840779 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.840747    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dzqk\" (UniqueName: \"kubernetes.io/projected/4c2572c2-6ab4-4e15-9497-8f8cb707c012-kube-api-access-6dzqk\") pod \"cert-manager-cainjector-559cdd8c87-j777b\" (UID: \"4c2572c2-6ab4-4e15-9497-8f8cb707c012\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:58.840895 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.840792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c2572c2-6ab4-4e15-9497-8f8cb707c012-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j777b\" (UID: \"4c2572c2-6ab4-4e15-9497-8f8cb707c012\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:58.858634 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.858609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4c2572c2-6ab4-4e15-9497-8f8cb707c012-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-j777b\" (UID: \"4c2572c2-6ab4-4e15-9497-8f8cb707c012\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:58.859099 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.859082    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dzqk\" (UniqueName: \"kubernetes.io/projected/4c2572c2-6ab4-4e15-9497-8f8cb707c012-kube-api-access-6dzqk\") pod \"cert-manager-cainjector-559cdd8c87-j777b\" (UID: \"4c2572c2-6ab4-4e15-9497-8f8cb707c012\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:58.958670 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:58.958581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b"
Jul 21 12:31:59.086827 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:59.086788    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-j777b"]
Jul 21 12:31:59.090860 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:31:59.090834    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c2572c2_6ab4_4e15_9497_8f8cb707c012.slice/crio-8c475495916bec8da1b0a59de6f24d232fc8400961bb6d27c73743c7475c670f WatchSource:0}: Error finding container 8c475495916bec8da1b0a59de6f24d232fc8400961bb6d27c73743c7475c670f: Status 404 returned error can't find the container with id 8c475495916bec8da1b0a59de6f24d232fc8400961bb6d27c73743c7475c670f
Jul 21 12:31:59.766748 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:31:59.766712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b" event={"ID":"4c2572c2-6ab4-4e15-9497-8f8cb707c012","Type":"ContainerStarted","Data":"8c475495916bec8da1b0a59de6f24d232fc8400961bb6d27c73743c7475c670f"}
Jul 21 12:32:02.778684 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:02.778651    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b" event={"ID":"4c2572c2-6ab4-4e15-9497-8f8cb707c012","Type":"ContainerStarted","Data":"2bdd03bc7af28fad2d82a6ca869c2a64474d8f72745ad6cbcabb984baeb1f346"}
Jul 21 12:32:02.798223 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:02.798174    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-j777b" podStartSLOduration=1.895363604 podStartE2EDuration="4.798159136s" totalImagesPullingTime="2.902795532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:31:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:02.796185923 +0000 UTC m=+260.581200888" watchObservedRunningTime="2026-07-21 12:32:02.798159136 +0000 UTC m=+260.583174100"
Jul 21 12:32:09.192767 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.192689    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"]
Jul 21 12:32:09.194818 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.194803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.198522 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.198502    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 21 12:32:09.198620 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.198588    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-x74t8\""
Jul 21 12:32:09.199451 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.199437    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 21 12:32:09.225700 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.225680    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"]
Jul 21 12:32:09.313568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.313542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e22ed42-bf05-432a-b045-3506cb8b144c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rscbm\" (UID: \"0e22ed42-bf05-432a-b045-3506cb8b144c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.313665 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.313573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjx2b\" (UniqueName: \"kubernetes.io/projected/0e22ed42-bf05-432a-b045-3506cb8b144c-kube-api-access-zjx2b\") pod \"openshift-lws-operator-fd8ccff4c-rscbm\" (UID: \"0e22ed42-bf05-432a-b045-3506cb8b144c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.414226 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.414196    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e22ed42-bf05-432a-b045-3506cb8b144c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rscbm\" (UID: \"0e22ed42-bf05-432a-b045-3506cb8b144c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.414334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.414230    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjx2b\" (UniqueName: \"kubernetes.io/projected/0e22ed42-bf05-432a-b045-3506cb8b144c-kube-api-access-zjx2b\") pod \"openshift-lws-operator-fd8ccff4c-rscbm\" (UID: \"0e22ed42-bf05-432a-b045-3506cb8b144c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.414542 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.414526    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e22ed42-bf05-432a-b045-3506cb8b144c-tmp\") pod \"openshift-lws-operator-fd8ccff4c-rscbm\" (UID: \"0e22ed42-bf05-432a-b045-3506cb8b144c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.429355 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.429325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjx2b\" (UniqueName: \"kubernetes.io/projected/0e22ed42-bf05-432a-b045-3506cb8b144c-kube-api-access-zjx2b\") pod \"openshift-lws-operator-fd8ccff4c-rscbm\" (UID: \"0e22ed42-bf05-432a-b045-3506cb8b144c\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.504021 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.504001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"
Jul 21 12:32:09.633509 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.633478    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm"]
Jul 21 12:32:09.636819 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:32:09.636793    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e22ed42_bf05_432a_b045_3506cb8b144c.slice/crio-59731a53241beeee0479c4806ea9bd1ffb8a38d322996373699470a1fa36656f WatchSource:0}: Error finding container 59731a53241beeee0479c4806ea9bd1ffb8a38d322996373699470a1fa36656f: Status 404 returned error can't find the container with id 59731a53241beeee0479c4806ea9bd1ffb8a38d322996373699470a1fa36656f
Jul 21 12:32:09.802669 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:09.802595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm" event={"ID":"0e22ed42-bf05-432a-b045-3506cb8b144c","Type":"ContainerStarted","Data":"59731a53241beeee0479c4806ea9bd1ffb8a38d322996373699470a1fa36656f"}
Jul 21 12:32:12.816861 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:12.816826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm" event={"ID":"0e22ed42-bf05-432a-b045-3506cb8b144c","Type":"ContainerStarted","Data":"ebbd38be8167f7c981f291c5ce384635fb5aac5844c6fc272814232ffb9b521b"}
Jul 21 12:32:28.559856 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.559798    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-rscbm" podStartSLOduration=17.197583438 podStartE2EDuration="19.559782584s" totalImagesPullingTime="2.362199146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:12.850836724 +0000 UTC m=+270.635851691" watchObservedRunningTime="2026-07-21 12:32:28.559782584 +0000 UTC m=+286.344797550"
Jul 21 12:32:28.560328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.560298    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"]
Jul 21 12:32:28.568533 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.568510    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.573536 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.573513    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 21 12:32:28.573692 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.573674    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 21 12:32:28.574029 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.574014    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 21 12:32:28.575154 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.575137    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-49kkt\""
Jul 21 12:32:28.576171 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.576152    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 21 12:32:28.584629 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.584610    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"]
Jul 21 12:32:28.649595 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.649568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd460f8b-5eee-498e-b818-5c78666719dc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.649746 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.649606    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd460f8b-5eee-498e-b818-5c78666719dc-tmp\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.649746 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.649626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fd460f8b-5eee-498e-b818-5c78666719dc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.649746 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.649696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fd460f8b-5eee-498e-b818-5c78666719dc-manifests\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.649746 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.649722    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6sxj\" (UniqueName: \"kubernetes.io/projected/fd460f8b-5eee-498e-b818-5c78666719dc-kube-api-access-k6sxj\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.750360 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.750330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd460f8b-5eee-498e-b818-5c78666719dc-tmp\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.750360 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.750361    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fd460f8b-5eee-498e-b818-5c78666719dc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.750589 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.750385    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fd460f8b-5eee-498e-b818-5c78666719dc-manifests\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.750589 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.750400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6sxj\" (UniqueName: \"kubernetes.io/projected/fd460f8b-5eee-498e-b818-5c78666719dc-kube-api-access-k6sxj\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.750589 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.750439    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd460f8b-5eee-498e-b818-5c78666719dc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.750839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.750819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/fd460f8b-5eee-498e-b818-5c78666719dc-manifests\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.752601 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.752572    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fd460f8b-5eee-498e-b818-5c78666719dc-tmp\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.752874 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.752856    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/fd460f8b-5eee-498e-b818-5c78666719dc-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.752938 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.752900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/fd460f8b-5eee-498e-b818-5c78666719dc-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.764601 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.764579    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6sxj\" (UniqueName: \"kubernetes.io/projected/fd460f8b-5eee-498e-b818-5c78666719dc-kube-api-access-k6sxj\") pod \"opendatahub-operator-controller-manager-7cb6668c8b-zbxvm\" (UID: \"fd460f8b-5eee-498e-b818-5c78666719dc\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:28.878500 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:28.878418    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:29.016270 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:29.016237    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"]
Jul 21 12:32:29.020673 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:32:29.020640    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd460f8b_5eee_498e_b818_5c78666719dc.slice/crio-d315c8e7a4a4d2e84870bc256f4a6a85ca1dd40b4cb9bf4dc345e91eeb0f04e8 WatchSource:0}: Error finding container d315c8e7a4a4d2e84870bc256f4a6a85ca1dd40b4cb9bf4dc345e91eeb0f04e8: Status 404 returned error can't find the container with id d315c8e7a4a4d2e84870bc256f4a6a85ca1dd40b4cb9bf4dc345e91eeb0f04e8
Jul 21 12:32:29.875804 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:29.875760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm" event={"ID":"fd460f8b-5eee-498e-b818-5c78666719dc","Type":"ContainerStarted","Data":"d315c8e7a4a4d2e84870bc256f4a6a85ca1dd40b4cb9bf4dc345e91eeb0f04e8"}
Jul 21 12:32:32.888148 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:32.888067    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm" event={"ID":"fd460f8b-5eee-498e-b818-5c78666719dc","Type":"ContainerStarted","Data":"f5b91a907de59a9e458a548ecbc6323cc6293d4e98ff3622cf89e2bc27c9ce41"}
Jul 21 12:32:33.896490 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:33.896458    2577 generic.go:356] "Generic (PLEG): container finished" podID="fd460f8b-5eee-498e-b818-5c78666719dc" containerID="f5b91a907de59a9e458a548ecbc6323cc6293d4e98ff3622cf89e2bc27c9ce41" exitCode=0
Jul 21 12:32:33.896879 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:33.896513    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm" event={"ID":"fd460f8b-5eee-498e-b818-5c78666719dc","Type":"ContainerDied","Data":"f5b91a907de59a9e458a548ecbc6323cc6293d4e98ff3622cf89e2bc27c9ce41"}
Jul 21 12:32:34.901622 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:34.901586    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm" event={"ID":"fd460f8b-5eee-498e-b818-5c78666719dc","Type":"ContainerStarted","Data":"6ce408a213567fee62995f34b017a09e744edc858411b1a1b7c119e8299a3863"}
Jul 21 12:32:34.902035 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:34.901721    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:34.959591 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:34.959529    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm" podStartSLOduration=1.734016368 podStartE2EDuration="6.959511832s" totalImagesPullingTime="4.141310974s" totalInitContainerRuntime="1.08418449s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:28 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:34.959222708 +0000 UTC m=+292.744237710" watchObservedRunningTime="2026-07-21 12:32:34.959511832 +0000 UTC m=+292.744526798"
Jul 21 12:32:42.633991 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:42.633948    2577 kubelet.go:1648] "Image garbage collection succeeded"
Jul 21 12:32:45.907561 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:45.907534    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb6668c8b-zbxvm"
Jul 21 12:32:48.946844 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:48.946811    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xb84h"]
Jul 21 12:32:48.949023 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:48.949002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:48.951733 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:48.951714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 21 12:32:48.951895 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:48.951879    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ndt9l\""
Jul 21 12:32:48.975947 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:48.975925    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xb84h"]
Jul 21 12:32:48.994828 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:48.994806    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv4kf\" (UniqueName: \"kubernetes.io/projected/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-kube-api-access-rv4kf\") pod \"odh-model-controller-7998c67dc4-xb84h\" (UID: \"cadc5390-e3f0-4511-9d15-12d6a9bb3da3\") " pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:48.994919 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:48.994853    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-cert\") pod \"odh-model-controller-7998c67dc4-xb84h\" (UID: \"cadc5390-e3f0-4511-9d15-12d6a9bb3da3\") " pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:49.095876 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.095846    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-cert\") pod \"odh-model-controller-7998c67dc4-xb84h\" (UID: \"cadc5390-e3f0-4511-9d15-12d6a9bb3da3\") " pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:49.096044 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.095898    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv4kf\" (UniqueName: \"kubernetes.io/projected/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-kube-api-access-rv4kf\") pod \"odh-model-controller-7998c67dc4-xb84h\" (UID: \"cadc5390-e3f0-4511-9d15-12d6a9bb3da3\") " pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:49.096044 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:32:49.096021    2577 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 21 12:32:49.096187 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:32:49.096092    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-cert podName:cadc5390-e3f0-4511-9d15-12d6a9bb3da3 nodeName:}" failed. No retries permitted until 2026-07-21 12:32:49.596071718 +0000 UTC m=+307.381086664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-cert") pod "odh-model-controller-7998c67dc4-xb84h" (UID: "cadc5390-e3f0-4511-9d15-12d6a9bb3da3") : secret "odh-model-controller-webhook-cert" not found
Jul 21 12:32:49.107142 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.107116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv4kf\" (UniqueName: \"kubernetes.io/projected/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-kube-api-access-rv4kf\") pod \"odh-model-controller-7998c67dc4-xb84h\" (UID: \"cadc5390-e3f0-4511-9d15-12d6a9bb3da3\") " pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:49.600192 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.600156    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-cert\") pod \"odh-model-controller-7998c67dc4-xb84h\" (UID: \"cadc5390-e3f0-4511-9d15-12d6a9bb3da3\") " pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:49.602495 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.602470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cadc5390-e3f0-4511-9d15-12d6a9bb3da3-cert\") pod \"odh-model-controller-7998c67dc4-xb84h\" (UID: \"cadc5390-e3f0-4511-9d15-12d6a9bb3da3\") " pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:49.858498 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.858413    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:49.984410 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.984377    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c5gbn"]
Jul 21 12:32:49.987184 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.987163    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:49.989313 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:49.989292    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 21 12:32:50.006249 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.006225    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c5gbn"]
Jul 21 12:32:50.007803 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.007784    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-xb84h"]
Jul 21 12:32:50.009767 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:32:50.009730    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcadc5390_e3f0_4511_9d15_12d6a9bb3da3.slice/crio-029f4898e21a3bf133cf590d107b88b0b24f1cbb96070de5330234f93db9e322 WatchSource:0}: Error finding container 029f4898e21a3bf133cf590d107b88b0b24f1cbb96070de5330234f93db9e322: Status 404 returned error can't find the container with id 029f4898e21a3bf133cf590d107b88b0b24f1cbb96070de5330234f93db9e322
Jul 21 12:32:50.011437 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.011422    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:32:50.105206 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.105180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2782bae9-cb6a-435b-b17c-f620c3b7a70d-tls-certs\") pod \"model-serving-api-54d89bd5fc-c5gbn\" (UID: \"2782bae9-cb6a-435b-b17c-f620c3b7a70d\") " pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:50.105312 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.105236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkxrc\" (UniqueName: \"kubernetes.io/projected/2782bae9-cb6a-435b-b17c-f620c3b7a70d-kube-api-access-gkxrc\") pod \"model-serving-api-54d89bd5fc-c5gbn\" (UID: \"2782bae9-cb6a-435b-b17c-f620c3b7a70d\") " pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:50.206610 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.206548    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2782bae9-cb6a-435b-b17c-f620c3b7a70d-tls-certs\") pod \"model-serving-api-54d89bd5fc-c5gbn\" (UID: \"2782bae9-cb6a-435b-b17c-f620c3b7a70d\") " pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:50.206610 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.206598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkxrc\" (UniqueName: \"kubernetes.io/projected/2782bae9-cb6a-435b-b17c-f620c3b7a70d-kube-api-access-gkxrc\") pod \"model-serving-api-54d89bd5fc-c5gbn\" (UID: \"2782bae9-cb6a-435b-b17c-f620c3b7a70d\") " pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:50.208837 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.208820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2782bae9-cb6a-435b-b17c-f620c3b7a70d-tls-certs\") pod \"model-serving-api-54d89bd5fc-c5gbn\" (UID: \"2782bae9-cb6a-435b-b17c-f620c3b7a70d\") " pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:50.218339 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.218315    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkxrc\" (UniqueName: \"kubernetes.io/projected/2782bae9-cb6a-435b-b17c-f620c3b7a70d-kube-api-access-gkxrc\") pod \"model-serving-api-54d89bd5fc-c5gbn\" (UID: \"2782bae9-cb6a-435b-b17c-f620c3b7a70d\") " pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:50.296653 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.296620    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:50.428506 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.428480    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-c5gbn"]
Jul 21 12:32:50.430513 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:32:50.430490    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2782bae9_cb6a_435b_b17c_f620c3b7a70d.slice/crio-7f25e4982b602d283a28f86c7f3d4438bd734bfcd434715c281823edb3ee0623 WatchSource:0}: Error finding container 7f25e4982b602d283a28f86c7f3d4438bd734bfcd434715c281823edb3ee0623: Status 404 returned error can't find the container with id 7f25e4982b602d283a28f86c7f3d4438bd734bfcd434715c281823edb3ee0623
Jul 21 12:32:50.675565 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.675527    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"]
Jul 21 12:32:50.678064 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.678040    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.680430 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.680408    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 21 12:32:50.681087 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.680938    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 21 12:32:50.681233 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.681192    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 21 12:32:50.681299 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.681276    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vh25j\""
Jul 21 12:32:50.681372 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.681353    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 21 12:32:50.690263 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.690238    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"]
Jul 21 12:32:50.811702 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.811667    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8be54e4-130a-41d2-8b31-d27d3d1754c9-tls-certs\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.811880 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.811727    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jf5f\" (UniqueName: \"kubernetes.io/projected/e8be54e4-130a-41d2-8b31-d27d3d1754c9-kube-api-access-9jf5f\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.811880 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.811780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8be54e4-130a-41d2-8b31-d27d3d1754c9-tmp\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.912985 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.912943    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jf5f\" (UniqueName: \"kubernetes.io/projected/e8be54e4-130a-41d2-8b31-d27d3d1754c9-kube-api-access-9jf5f\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.913169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.913006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8be54e4-130a-41d2-8b31-d27d3d1754c9-tmp\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.913169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.913080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8be54e4-130a-41d2-8b31-d27d3d1754c9-tls-certs\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.915554 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.915523    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e8be54e4-130a-41d2-8b31-d27d3d1754c9-tmp\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.917086 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.917061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e8be54e4-130a-41d2-8b31-d27d3d1754c9-tls-certs\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.924020 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.923995    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jf5f\" (UniqueName: \"kubernetes.io/projected/e8be54e4-130a-41d2-8b31-d27d3d1754c9-kube-api-access-9jf5f\") pod \"kube-auth-proxy-5b686cc986-z6bnl\" (UID: \"e8be54e4-130a-41d2-8b31-d27d3d1754c9\") " pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:50.958473 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.958404    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn" event={"ID":"2782bae9-cb6a-435b-b17c-f620c3b7a70d","Type":"ContainerStarted","Data":"7f25e4982b602d283a28f86c7f3d4438bd734bfcd434715c281823edb3ee0623"}
Jul 21 12:32:50.959731 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.959707    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xb84h" event={"ID":"cadc5390-e3f0-4511-9d15-12d6a9bb3da3","Type":"ContainerStarted","Data":"029f4898e21a3bf133cf590d107b88b0b24f1cbb96070de5330234f93db9e322"}
Jul 21 12:32:50.991770 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:50.991743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"
Jul 21 12:32:51.166308 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.166168    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl"]
Jul 21 12:32:51.169013 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:32:51.168930    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8be54e4_130a_41d2_8b31_d27d3d1754c9.slice/crio-c59c53183c636116517ffd549bc44fe803dda11ae942fc7c19a2aee06c853835 WatchSource:0}: Error finding container c59c53183c636116517ffd549bc44fe803dda11ae942fc7c19a2aee06c853835: Status 404 returned error can't find the container with id c59c53183c636116517ffd549bc44fe803dda11ae942fc7c19a2aee06c853835
Jul 21 12:32:51.912492 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.912464    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"]
Jul 21 12:32:51.916414 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.916390    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:51.918884 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.918707    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 21 12:32:51.918884 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.918722    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 21 12:32:51.918884 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.918729    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 21 12:32:51.918884 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.918729    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 21 12:32:51.919193 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.919094    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fn49f\""
Jul 21 12:32:51.929800 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.929765    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"]
Jul 21 12:32:51.966606 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:51.966571    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl" event={"ID":"e8be54e4-130a-41d2-8b31-d27d3d1754c9","Type":"ContainerStarted","Data":"c59c53183c636116517ffd549bc44fe803dda11ae942fc7c19a2aee06c853835"}
Jul 21 12:32:52.021448 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.021418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rp59x\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-kube-api-access-rp59x\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.021848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.021469    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-local-certs\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.021848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.021511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.021848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.021548    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.021848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.021593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-token\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.021848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.021645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-cacerts\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.021848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.021679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.122774 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.122741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rp59x\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-kube-api-access-rp59x\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.122950 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.122797    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-local-certs\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.122950 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.122832    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.122950 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.122854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.122950 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.122884    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-token\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.123191 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.122969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-cacerts\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.123191 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.123032    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.123626 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.123598    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.125408 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.125386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-local-certs\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.125634 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.125593    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.125767 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.125746    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-cacerts\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.125818 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.125772    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.130944 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.130923    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rp59x\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-kube-api-access-rp59x\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.131183 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.131165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-token\") pod \"istiod-openshift-gateway-7574459656-92c8v\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:52.227397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:52.227308    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:53.175692 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:53.175616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"]
Jul 21 12:32:53.178578 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:32:53.178548    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeffcacc3_bc0e_4b79_b297_ae24b7cd8a9a.slice/crio-ebf2b3200e9e4cffc62f042ed787ef746e6cf5a1a40f613a2b3e71bb3a5c5a9a WatchSource:0}: Error finding container ebf2b3200e9e4cffc62f042ed787ef746e6cf5a1a40f613a2b3e71bb3a5c5a9a: Status 404 returned error can't find the container with id ebf2b3200e9e4cffc62f042ed787ef746e6cf5a1a40f613a2b3e71bb3a5c5a9a
Jul 21 12:32:53.978941 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:53.978902    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn" event={"ID":"2782bae9-cb6a-435b-b17c-f620c3b7a70d","Type":"ContainerStarted","Data":"e98adb6d4c7109c9d7646b96b4af96f84568249b3035b1e7a7fcc90e1ebe9364"}
Jul 21 12:32:53.979160 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:53.979066    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:32:53.980828 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:53.980794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-xb84h" event={"ID":"cadc5390-e3f0-4511-9d15-12d6a9bb3da3","Type":"ContainerStarted","Data":"2d1244303d4ebce20669c1ab8c7a996ef6a7bb0e55ec49997598abee204923f9"}
Jul 21 12:32:53.980948 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:53.980885    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:32:53.982326 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:53.982300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v" event={"ID":"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a","Type":"ContainerStarted","Data":"ebf2b3200e9e4cffc62f042ed787ef746e6cf5a1a40f613a2b3e71bb3a5c5a9a"}
Jul 21 12:32:53.997568 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:53.997508    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn" podStartSLOduration=2.406754112 podStartE2EDuration="4.997495677s" totalImagesPullingTime="2.590741565s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:53.996076001 +0000 UTC m=+311.781090991" watchObservedRunningTime="2026-07-21 12:32:53.997495677 +0000 UTC m=+311.782510643"
Jul 21 12:32:54.016679 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:54.016637    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-xb84h" podStartSLOduration=3.000894577 podStartE2EDuration="6.016624874s" totalImagesPullingTime="3.015730297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:54.015518268 +0000 UTC m=+311.800533235" watchObservedRunningTime="2026-07-21 12:32:54.016624874 +0000 UTC m=+311.801639840"
Jul 21 12:32:54.988389 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:54.988237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl" event={"ID":"e8be54e4-130a-41d2-8b31-d27d3d1754c9","Type":"ContainerStarted","Data":"9b4407ebd2b5cc70959190d3b224b3b2df68df33f038aed9d3483d7f6e7d91fa"}
Jul 21 12:32:55.007080 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:55.007022    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b686cc986-z6bnl" podStartSLOduration=2.070642078 podStartE2EDuration="5.007001103s" totalImagesPullingTime="2.936359025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:55.004184214 +0000 UTC m=+312.789199207" watchObservedRunningTime="2026-07-21 12:32:55.007001103 +0000 UTC m=+312.792016069"
Jul 21 12:32:55.880450 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:55.880417    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:32:55.880526 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:55.880483    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:32:55.993807 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:55.993771    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v" event={"ID":"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a","Type":"ContainerStarted","Data":"db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e"}
Jul 21 12:32:55.994297 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:55.994059    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:32:56.033124 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:56.033061    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v" podStartSLOduration=2.333730989 podStartE2EDuration="5.033041736s" totalImagesPullingTime="2.699310747s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:32:56.031706641 +0000 UTC m=+313.816721609" watchObservedRunningTime="2026-07-21 12:32:56.033041736 +0000 UTC m=+313.818056703"
Jul 21 12:32:56.999378 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:56.999341    2577 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-92c8v container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 12:32:56.999710 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:56.999394    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v" podUID="effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 12:32:57.505779 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.505746    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"]
Jul 21 12:32:57.509072 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.509052    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.511425 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.511403    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-q9mjp\""
Jul 21 12:32:57.527035 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.527014    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"]
Jul 21 12:32:57.673818 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.673782    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.673818 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.673817    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbn29\" (UniqueName: \"kubernetes.io/projected/c28b5daa-f9df-41d9-b524-dea8642cd8d1-kube-api-access-mbn29\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.674082 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.673904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.674082 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.673936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.674082 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.674014    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.674082 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.674041    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.674082 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.674063    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.674336 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.674090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.674336 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.674110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775348 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775348 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775348 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775392    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775411    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775433    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775494    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbn29\" (UniqueName: \"kubernetes.io/projected/c28b5daa-f9df-41d9-b524-dea8642cd8d1-kube-api-access-mbn29\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.775923 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775894    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.776092 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.776067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.776237 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.775919    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.776237 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.776142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.776394 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.776283    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.777753 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.777734    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.778009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.777992    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.790008 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.789985    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c28b5daa-f9df-41d9-b524-dea8642cd8d1-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.790462 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.790441    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbn29\" (UniqueName: \"kubernetes.io/projected/c28b5daa-f9df-41d9-b524-dea8642cd8d1-kube-api-access-mbn29\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678g2wxn\" (UID: \"c28b5daa-f9df-41d9-b524-dea8642cd8d1\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.819452 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.819426    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:32:57.966258 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:57.966225    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"]
Jul 21 12:32:57.968846 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:32:57.968816    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc28b5daa_f9df_41d9_b524_dea8642cd8d1.slice/crio-de7e44cd29dbaf41f91eb29b97e059530b3fdb17f8cc06d2132f498e972fc206 WatchSource:0}: Error finding container de7e44cd29dbaf41f91eb29b97e059530b3fdb17f8cc06d2132f498e972fc206: Status 404 returned error can't find the container with id de7e44cd29dbaf41f91eb29b97e059530b3fdb17f8cc06d2132f498e972fc206
Jul 21 12:32:58.007095 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:58.007066    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn" event={"ID":"c28b5daa-f9df-41d9-b524-dea8642cd8d1","Type":"ContainerStarted","Data":"de7e44cd29dbaf41f91eb29b97e059530b3fdb17f8cc06d2132f498e972fc206"}
Jul 21 12:32:59.999142 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:32:59.999107    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:33:00.359892 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.359815    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"]
Jul 21 12:33:00.366747 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.366728    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.369351 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.369327    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"]
Jul 21 12:33:00.370013 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.369988    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 21 12:33:00.370354 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.370339    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xj45j\""
Jul 21 12:33:00.372592 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.372572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.375158 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.375007    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 21 12:33:00.375158 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.375118    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-d6hqv\""
Jul 21 12:33:00.390630 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.390545    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"]
Jul 21 12:33:00.392326 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.392301    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"]
Jul 21 12:33:00.500249 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.500217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xn265\" (UniqueName: \"kubernetes.io/projected/8e044a3a-dd4c-44fd-bf33-feb5494e5dfe-kube-api-access-xn265\") pod \"kserve-controller-manager-d6f4b65b8-f29z8\" (UID: \"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.500405 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.500258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e044a3a-dd4c-44fd-bf33-feb5494e5dfe-cert\") pod \"kserve-controller-manager-d6f4b65b8-f29z8\" (UID: \"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.500405 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.500300    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d472edf-63c8-4c08-a6a0-99b82436388d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zw2vt\" (UID: \"3d472edf-63c8-4c08-a6a0-99b82436388d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.500405 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.500383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zn4gs\" (UniqueName: \"kubernetes.io/projected/3d472edf-63c8-4c08-a6a0-99b82436388d-kube-api-access-zn4gs\") pod \"llmisvc-controller-manager-777c6bd7d5-zw2vt\" (UID: \"3d472edf-63c8-4c08-a6a0-99b82436388d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.601499 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.601458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d472edf-63c8-4c08-a6a0-99b82436388d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zw2vt\" (UID: \"3d472edf-63c8-4c08-a6a0-99b82436388d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.601703 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.601531    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zn4gs\" (UniqueName: \"kubernetes.io/projected/3d472edf-63c8-4c08-a6a0-99b82436388d-kube-api-access-zn4gs\") pod \"llmisvc-controller-manager-777c6bd7d5-zw2vt\" (UID: \"3d472edf-63c8-4c08-a6a0-99b82436388d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.601703 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.601603    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xn265\" (UniqueName: \"kubernetes.io/projected/8e044a3a-dd4c-44fd-bf33-feb5494e5dfe-kube-api-access-xn265\") pod \"kserve-controller-manager-d6f4b65b8-f29z8\" (UID: \"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.601703 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.601645    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e044a3a-dd4c-44fd-bf33-feb5494e5dfe-cert\") pod \"kserve-controller-manager-d6f4b65b8-f29z8\" (UID: \"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.604199 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.604172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3d472edf-63c8-4c08-a6a0-99b82436388d-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-zw2vt\" (UID: \"3d472edf-63c8-4c08-a6a0-99b82436388d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.604300 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.604277    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8e044a3a-dd4c-44fd-bf33-feb5494e5dfe-cert\") pod \"kserve-controller-manager-d6f4b65b8-f29z8\" (UID: \"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.622906 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.622844    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zn4gs\" (UniqueName: \"kubernetes.io/projected/3d472edf-63c8-4c08-a6a0-99b82436388d-kube-api-access-zn4gs\") pod \"llmisvc-controller-manager-777c6bd7d5-zw2vt\" (UID: \"3d472edf-63c8-4c08-a6a0-99b82436388d\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.624471 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.624446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xn265\" (UniqueName: \"kubernetes.io/projected/8e044a3a-dd4c-44fd-bf33-feb5494e5dfe-kube-api-access-xn265\") pod \"kserve-controller-manager-d6f4b65b8-f29z8\" (UID: \"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.680144 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.680116    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:00.687419 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.687392    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:00.778280 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.778236    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:33:00.778389 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.778305    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:33:00.778389 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.778333    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:33:00.861560 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.861540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"]
Jul 21 12:33:00.863481 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:33:00.863454    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e044a3a_dd4c_44fd_bf33_feb5494e5dfe.slice/crio-e71de3a8513eda1181d9641de59bf7955906ab1a06043cb7c7a1112fd3f7c3a9 WatchSource:0}: Error finding container e71de3a8513eda1181d9641de59bf7955906ab1a06043cb7c7a1112fd3f7c3a9: Status 404 returned error can't find the container with id e71de3a8513eda1181d9641de59bf7955906ab1a06043cb7c7a1112fd3f7c3a9
Jul 21 12:33:00.869538 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:00.869518    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"]
Jul 21 12:33:00.872449 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:33:00.872429    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3d472edf_63c8_4c08_a6a0_99b82436388d.slice/crio-58441886ce1bb21ccf9bf172267749bf1b333f10dddb565ed6abcef7ab6934ec WatchSource:0}: Error finding container 58441886ce1bb21ccf9bf172267749bf1b333f10dddb565ed6abcef7ab6934ec: Status 404 returned error can't find the container with id 58441886ce1bb21ccf9bf172267749bf1b333f10dddb565ed6abcef7ab6934ec
Jul 21 12:33:01.022516 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:01.022484    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt" event={"ID":"3d472edf-63c8-4c08-a6a0-99b82436388d","Type":"ContainerStarted","Data":"58441886ce1bb21ccf9bf172267749bf1b333f10dddb565ed6abcef7ab6934ec"}
Jul 21 12:33:01.024102 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:01.024071    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn" event={"ID":"c28b5daa-f9df-41d9-b524-dea8642cd8d1","Type":"ContainerStarted","Data":"26842e804ac17d5fa3a83f02dd7e21dd256012c99d4dbb26ccb2dfd088535e3d"}
Jul 21 12:33:01.025157 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:01.025135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8" event={"ID":"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe","Type":"ContainerStarted","Data":"e71de3a8513eda1181d9641de59bf7955906ab1a06043cb7c7a1112fd3f7c3a9"}
Jul 21 12:33:01.078762 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:01.078717    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn" podStartSLOduration=1.271515181 podStartE2EDuration="4.078703897s" totalImagesPullingTime="2.807188716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:32:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:33:01.077396625 +0000 UTC m=+318.862411593" watchObservedRunningTime="2026-07-21 12:33:01.078703897 +0000 UTC m=+318.863718862"
Jul 21 12:33:01.819910 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:01.819863    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:33:01.825315 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:01.825288    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:33:02.030365 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:02.030333    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:33:02.031598 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:02.031574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678g2wxn"
Jul 21 12:33:04.040805 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.040775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt" event={"ID":"3d472edf-63c8-4c08-a6a0-99b82436388d","Type":"ContainerStarted","Data":"df2a0ee04d96bdadacee0f0188e71233c3820db722c27aaa1aebc40bbd410abc"}
Jul 21 12:33:04.041236 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.040839    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:04.042361 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.042340    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8" event={"ID":"8e044a3a-dd4c-44fd-bf33-feb5494e5dfe","Type":"ContainerStarted","Data":"cba2aa9f2e6a9ccfe4f896e8c8fa11ec7bb648291c73a4224fd2e2cf305b4027"}
Jul 21 12:33:04.042555 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.042543    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:04.080816 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.080743    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt" podStartSLOduration=1.135569909 podStartE2EDuration="4.080729766s" totalImagesPullingTime="2.945159857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:33:04.079469239 +0000 UTC m=+321.864484204" watchObservedRunningTime="2026-07-21 12:33:04.080729766 +0000 UTC m=+321.865744731"
Jul 21 12:33:04.103299 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.103256    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8" podStartSLOduration=1.1537590739999999 podStartE2EDuration="4.103243102s" totalImagesPullingTime="2.949484028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:33:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:33:04.102639611 +0000 UTC m=+321.887654575" watchObservedRunningTime="2026-07-21 12:33:04.103243102 +0000 UTC m=+321.888258068"
Jul 21 12:33:04.497663 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.497630    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"]
Jul 21 12:33:04.497869 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.497843    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v" podUID="effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" containerName="discovery" containerID="cri-o://db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e" gracePeriod=30
Jul 21 12:33:04.785600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.785575    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:33:04.937347 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937316    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-cacerts\") pod \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") "
Jul 21 12:33:04.937499 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937377    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-token\") pod \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") "
Jul 21 12:33:04.937499 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937420    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-ca-configmap\") pod \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") "
Jul 21 12:33:04.937499 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937459    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-local-certs\") pod \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") "
Jul 21 12:33:04.937616 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937525    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rp59x\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-kube-api-access-rp59x\") pod \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") "
Jul 21 12:33:04.937616 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937552    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-kubeconfig\") pod \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") "
Jul 21 12:33:04.937616 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937573    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-dns-cert\") pod \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\" (UID: \"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a\") "
Jul 21 12:33:04.938065 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.937847    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" (UID: "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:33:04.940150 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.940119    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" (UID: "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:33:04.940150 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.940130    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" (UID: "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:33:04.940150 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.940141    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-local-certs" (OuterVolumeSpecName: "local-certs") pod "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" (UID: "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:33:04.940364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.940159    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" (UID: "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:33:04.940364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.940249    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-kube-api-access-rp59x" (OuterVolumeSpecName: "kube-api-access-rp59x") pod "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" (UID: "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a"). InnerVolumeSpecName "kube-api-access-rp59x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:33:04.940364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.940306    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-cacerts" (OuterVolumeSpecName: "cacerts") pod "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" (UID: "effcacc3-bc0e-4b79-b297-ae24b7cd8a9a"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:33:04.990849 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.990823    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-xb84h"
Jul 21 12:33:04.992685 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:04.992667    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-c5gbn"
Jul 21 12:33:05.038352 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.038290    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rp59x\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-kube-api-access-rp59x\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:05.038352 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.038314    2577 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-kubeconfig\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:05.038352 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.038323    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-dns-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:05.038352 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.038333    2577 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-cacerts\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:05.038352 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.038342    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:05.038352 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.038351    2577 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-istio-csr-ca-configmap\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:05.038595 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.038358    2577 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a-local-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:05.049033 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.049011    2577 generic.go:356] "Generic (PLEG): container finished" podID="effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" containerID="db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e" exitCode=0
Jul 21 12:33:05.049397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.049075    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"
Jul 21 12:33:05.049397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.049080    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v" event={"ID":"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a","Type":"ContainerDied","Data":"db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e"}
Jul 21 12:33:05.049397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.049133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-92c8v" event={"ID":"effcacc3-bc0e-4b79-b297-ae24b7cd8a9a","Type":"ContainerDied","Data":"ebf2b3200e9e4cffc62f042ed787ef746e6cf5a1a40f613a2b3e71bb3a5c5a9a"}
Jul 21 12:33:05.049906 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.049886    2577 scope.go:117] "RemoveContainer" containerID="db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e"
Jul 21 12:33:05.058706 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.058664    2577 scope.go:117] "RemoveContainer" containerID="db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e"
Jul 21 12:33:05.059013 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:33:05.058977    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e\": container with ID starting with db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e not found: ID does not exist" containerID="db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e"
Jul 21 12:33:05.059107 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.059055    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e"} err="failed to get container status \"db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e\": rpc error: code = NotFound desc = could not find container \"db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e\": container with ID starting with db5e4d352665163e4922ed0ca677c530f0f4f48ec097d2a86f1775f8a3ff2b2e not found: ID does not exist"
Jul 21 12:33:05.073342 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.073322    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"]
Jul 21 12:33:05.076572 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:05.076555    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-92c8v"]
Jul 21 12:33:06.832821 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:06.832789    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" path="/var/lib/kubelet/pods/effcacc3-bc0e-4b79-b297-ae24b7cd8a9a/volumes"
Jul 21 12:33:35.051653 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:35.051568    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-zw2vt"
Jul 21 12:33:35.054650 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:35.054627    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-f29z8"
Jul 21 12:33:49.805042 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.803539    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlrf9"]
Jul 21 12:33:49.805042 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.804240    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" containerName="discovery"
Jul 21 12:33:49.805042 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.804263    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" containerName="discovery"
Jul 21 12:33:49.805042 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.804424    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="effcacc3-bc0e-4b79-b297-ae24b7cd8a9a" containerName="discovery"
Jul 21 12:33:49.813145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.813120    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9"
Jul 21 12:33:49.816681 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.816654    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 21 12:33:49.816827 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.816740    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-5b5zq\""
Jul 21 12:33:49.816920 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.816902    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 21 12:33:49.817800 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.817776    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlrf9"]
Jul 21 12:33:49.879362 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.879331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldvhd\" (UniqueName: \"kubernetes.io/projected/8787f2a8-0c26-4ff2-90bd-47205b1fa226-kube-api-access-ldvhd\") pod \"kuadrant-operator-catalog-jlrf9\" (UID: \"8787f2a8-0c26-4ff2-90bd-47205b1fa226\") " pod="kuadrant-system/kuadrant-operator-catalog-jlrf9"
Jul 21 12:33:49.980140 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.980115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldvhd\" (UniqueName: \"kubernetes.io/projected/8787f2a8-0c26-4ff2-90bd-47205b1fa226-kube-api-access-ldvhd\") pod \"kuadrant-operator-catalog-jlrf9\" (UID: \"8787f2a8-0c26-4ff2-90bd-47205b1fa226\") " pod="kuadrant-system/kuadrant-operator-catalog-jlrf9"
Jul 21 12:33:49.989643 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:49.989621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldvhd\" (UniqueName: \"kubernetes.io/projected/8787f2a8-0c26-4ff2-90bd-47205b1fa226-kube-api-access-ldvhd\") pod \"kuadrant-operator-catalog-jlrf9\" (UID: \"8787f2a8-0c26-4ff2-90bd-47205b1fa226\") " pod="kuadrant-system/kuadrant-operator-catalog-jlrf9"
Jul 21 12:33:50.124625 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.124549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9"
Jul 21 12:33:50.159133 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.159100    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlrf9"]
Jul 21 12:33:50.248607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.248475    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlrf9"]
Jul 21 12:33:50.250721 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:33:50.250694    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8787f2a8_0c26_4ff2_90bd_47205b1fa226.slice/crio-002a0bc638b8f3d70a0bac940165ebc11c9a02ec6853ffff5f5d8708acb75d89 WatchSource:0}: Error finding container 002a0bc638b8f3d70a0bac940165ebc11c9a02ec6853ffff5f5d8708acb75d89: Status 404 returned error can't find the container with id 002a0bc638b8f3d70a0bac940165ebc11c9a02ec6853ffff5f5d8708acb75d89
Jul 21 12:33:50.368667 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.368635    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-phwn2"]
Jul 21 12:33:50.371579 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.371563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:33:50.379080 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.379059    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-phwn2"]
Jul 21 12:33:50.484543 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.484511    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqfvs\" (UniqueName: \"kubernetes.io/projected/2e701718-8ca7-4f2a-9914-1528c85cee0a-kube-api-access-fqfvs\") pod \"kuadrant-operator-catalog-phwn2\" (UID: \"2e701718-8ca7-4f2a-9914-1528c85cee0a\") " pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:33:50.585211 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.585183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqfvs\" (UniqueName: \"kubernetes.io/projected/2e701718-8ca7-4f2a-9914-1528c85cee0a-kube-api-access-fqfvs\") pod \"kuadrant-operator-catalog-phwn2\" (UID: \"2e701718-8ca7-4f2a-9914-1528c85cee0a\") " pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:33:50.594313 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.594293    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqfvs\" (UniqueName: \"kubernetes.io/projected/2e701718-8ca7-4f2a-9914-1528c85cee0a-kube-api-access-fqfvs\") pod \"kuadrant-operator-catalog-phwn2\" (UID: \"2e701718-8ca7-4f2a-9914-1528c85cee0a\") " pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:33:50.680818 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.680753    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:33:50.806260 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:50.806233    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-phwn2"]
Jul 21 12:33:50.807706 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:33:50.807677    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e701718_8ca7_4f2a_9914_1528c85cee0a.slice/crio-9e621fbd2e9a113c768c4cd939b7299fb0988d86673e07967b5de8f3fa1c76da WatchSource:0}: Error finding container 9e621fbd2e9a113c768c4cd939b7299fb0988d86673e07967b5de8f3fa1c76da: Status 404 returned error can't find the container with id 9e621fbd2e9a113c768c4cd939b7299fb0988d86673e07967b5de8f3fa1c76da
Jul 21 12:33:51.233872 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:51.233841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-phwn2" event={"ID":"2e701718-8ca7-4f2a-9914-1528c85cee0a","Type":"ContainerStarted","Data":"9e621fbd2e9a113c768c4cd939b7299fb0988d86673e07967b5de8f3fa1c76da"}
Jul 21 12:33:51.235135 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:51.235108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9" event={"ID":"8787f2a8-0c26-4ff2-90bd-47205b1fa226","Type":"ContainerStarted","Data":"002a0bc638b8f3d70a0bac940165ebc11c9a02ec6853ffff5f5d8708acb75d89"}
Jul 21 12:33:53.245506 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.245471    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-phwn2" event={"ID":"2e701718-8ca7-4f2a-9914-1528c85cee0a","Type":"ContainerStarted","Data":"f72b9d6d3203ac447abc6cb83e311684fc3b97d452bf6f783b9d355a05389416"}
Jul 21 12:33:53.246927 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.246903    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9" event={"ID":"8787f2a8-0c26-4ff2-90bd-47205b1fa226","Type":"ContainerStarted","Data":"00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af"}
Jul 21 12:33:53.247069 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.246944    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9" podUID="8787f2a8-0c26-4ff2-90bd-47205b1fa226" containerName="registry-server" containerID="cri-o://00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af" gracePeriod=2
Jul 21 12:33:53.262871 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.262831    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-phwn2" podStartSLOduration=1.716203464 podStartE2EDuration="3.262816167s" totalImagesPullingTime="1.546612703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:33:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:33:53.261318038 +0000 UTC m=+371.046333031" watchObservedRunningTime="2026-07-21 12:33:53.262816167 +0000 UTC m=+371.047831132"
Jul 21 12:33:53.276598 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.276557    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9" podStartSLOduration=2.175725162 podStartE2EDuration="4.276540629s" totalImagesPullingTime="2.100815467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:33:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:33:53.27507312 +0000 UTC m=+371.060088086" watchObservedRunningTime="2026-07-21 12:33:53.276540629 +0000 UTC m=+371.061555594"
Jul 21 12:33:53.495939 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.495885    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9"
Jul 21 12:33:53.612544 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.612507    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldvhd\" (UniqueName: \"kubernetes.io/projected/8787f2a8-0c26-4ff2-90bd-47205b1fa226-kube-api-access-ldvhd\") pod \"8787f2a8-0c26-4ff2-90bd-47205b1fa226\" (UID: \"8787f2a8-0c26-4ff2-90bd-47205b1fa226\") "
Jul 21 12:33:53.614560 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.614529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8787f2a8-0c26-4ff2-90bd-47205b1fa226-kube-api-access-ldvhd" (OuterVolumeSpecName: "kube-api-access-ldvhd") pod "8787f2a8-0c26-4ff2-90bd-47205b1fa226" (UID: "8787f2a8-0c26-4ff2-90bd-47205b1fa226"). InnerVolumeSpecName "kube-api-access-ldvhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:33:53.714100 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:53.714068    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldvhd\" (UniqueName: \"kubernetes.io/projected/8787f2a8-0c26-4ff2-90bd-47205b1fa226-kube-api-access-ldvhd\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:33:54.252071 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.252043    2577 generic.go:356] "Generic (PLEG): container finished" podID="8787f2a8-0c26-4ff2-90bd-47205b1fa226" containerID="00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af" exitCode=0
Jul 21 12:33:54.252491 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.252096    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9"
Jul 21 12:33:54.252491 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.252130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9" event={"ID":"8787f2a8-0c26-4ff2-90bd-47205b1fa226","Type":"ContainerDied","Data":"00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af"}
Jul 21 12:33:54.252491 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.252179    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jlrf9" event={"ID":"8787f2a8-0c26-4ff2-90bd-47205b1fa226","Type":"ContainerDied","Data":"002a0bc638b8f3d70a0bac940165ebc11c9a02ec6853ffff5f5d8708acb75d89"}
Jul 21 12:33:54.252491 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.252199    2577 scope.go:117] "RemoveContainer" containerID="00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af"
Jul 21 12:33:54.260836 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.260821    2577 scope.go:117] "RemoveContainer" containerID="00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af"
Jul 21 12:33:54.261120 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:33:54.261100    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af\": container with ID starting with 00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af not found: ID does not exist" containerID="00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af"
Jul 21 12:33:54.261187 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.261131    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af"} err="failed to get container status \"00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af\": rpc error: code = NotFound desc = could not find container \"00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af\": container with ID starting with 00bef4b91098daf523a27dfd93d1581aaa7a44a8a3238361e5392d0c620e03af not found: ID does not exist"
Jul 21 12:33:54.272253 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.272225    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlrf9"]
Jul 21 12:33:54.276513 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.276491    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jlrf9"]
Jul 21 12:33:54.831891 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:33:54.831856    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8787f2a8-0c26-4ff2-90bd-47205b1fa226" path="/var/lib/kubelet/pods/8787f2a8-0c26-4ff2-90bd-47205b1fa226/volumes"
Jul 21 12:34:00.681687 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:00.681644    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:34:00.682101 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:00.681739    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:34:00.703782 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:00.703751    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:34:01.303810 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:01.303782    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-phwn2"
Jul 21 12:34:23.000200 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.000165    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"]
Jul 21 12:34:23.000665 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.000471    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8787f2a8-0c26-4ff2-90bd-47205b1fa226" containerName="registry-server"
Jul 21 12:34:23.000665 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.000482    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8787f2a8-0c26-4ff2-90bd-47205b1fa226" containerName="registry-server"
Jul 21 12:34:23.000665 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.000540    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8787f2a8-0c26-4ff2-90bd-47205b1fa226" containerName="registry-server"
Jul 21 12:34:23.003345 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.003327    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"
Jul 21 12:34:23.005866 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.005849    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zc7k7\""
Jul 21 12:34:23.021147 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.021123    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"]
Jul 21 12:34:23.051220 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.051194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f88p2\" (UniqueName: \"kubernetes.io/projected/ee0753d5-545e-4843-ad66-cffa4cc0d3ca-kube-api-access-f88p2\") pod \"authorino-operator-75cc8d5cb7-x8gz5\" (UID: \"ee0753d5-545e-4843-ad66-cffa4cc0d3ca\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"
Jul 21 12:34:23.152236 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.152198    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f88p2\" (UniqueName: \"kubernetes.io/projected/ee0753d5-545e-4843-ad66-cffa4cc0d3ca-kube-api-access-f88p2\") pod \"authorino-operator-75cc8d5cb7-x8gz5\" (UID: \"ee0753d5-545e-4843-ad66-cffa4cc0d3ca\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"
Jul 21 12:34:23.168839 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.168807    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f88p2\" (UniqueName: \"kubernetes.io/projected/ee0753d5-545e-4843-ad66-cffa4cc0d3ca-kube-api-access-f88p2\") pod \"authorino-operator-75cc8d5cb7-x8gz5\" (UID: \"ee0753d5-545e-4843-ad66-cffa4cc0d3ca\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"
Jul 21 12:34:23.313017 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.312910    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"
Jul 21 12:34:23.446814 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:23.446783    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"]
Jul 21 12:34:23.449878 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:34:23.449847    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee0753d5_545e_4843_ad66_cffa4cc0d3ca.slice/crio-216039541cf707f471826cebc1ab7081775ba4cb0e32c74b4fb4b17aed3f40a1 WatchSource:0}: Error finding container 216039541cf707f471826cebc1ab7081775ba4cb0e32c74b4fb4b17aed3f40a1: Status 404 returned error can't find the container with id 216039541cf707f471826cebc1ab7081775ba4cb0e32c74b4fb4b17aed3f40a1
Jul 21 12:34:24.381737 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:24.381660    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5" event={"ID":"ee0753d5-545e-4843-ad66-cffa4cc0d3ca","Type":"ContainerStarted","Data":"216039541cf707f471826cebc1ab7081775ba4cb0e32c74b4fb4b17aed3f40a1"}
Jul 21 12:34:26.392017 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:26.391976    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5" event={"ID":"ee0753d5-545e-4843-ad66-cffa4cc0d3ca","Type":"ContainerStarted","Data":"f2c0bfbd86b720392af4822222ee69f11cfe2e9afbc835a401b0193dc7869890"}
Jul 21 12:34:26.392432 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:26.392096    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"
Jul 21 12:34:26.412537 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:26.412484    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5" podStartSLOduration=2.086171291 podStartE2EDuration="4.412469227s" totalImagesPullingTime="2.326297936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:26.410757067 +0000 UTC m=+404.195772031" watchObservedRunningTime="2026-07-21 12:34:26.412469227 +0000 UTC m=+404.197484193"
Jul 21 12:34:27.104066 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.104030    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"]
Jul 21 12:34:27.107271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.107254    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.109494 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.109472    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d4b6n\""
Jul 21 12:34:27.126485 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.126456    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"]
Jul 21 12:34:27.185098 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.185065    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.185241 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.185106    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ts6b\" (UniqueName: \"kubernetes.io/projected/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-kube-api-access-4ts6b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.286053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.286019    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.286193 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.286064    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ts6b\" (UniqueName: \"kubernetes.io/projected/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-kube-api-access-4ts6b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.286405 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.286388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.297814 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.297787    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ts6b\" (UniqueName: \"kubernetes.io/projected/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-kube-api-access-4ts6b\") pod \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.416813 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.416729    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:27.545410 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:27.545334    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"]
Jul 21 12:34:27.549561 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:34:27.549529    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a1cc992_38e6_442b_9ba0_c394c9ab8c4d.slice/crio-a4b4e4c882b74213f06a219ae2f0450e6ff27ebe168274afcf67ab894ef835ba WatchSource:0}: Error finding container a4b4e4c882b74213f06a219ae2f0450e6ff27ebe168274afcf67ab894ef835ba: Status 404 returned error can't find the container with id a4b4e4c882b74213f06a219ae2f0450e6ff27ebe168274afcf67ab894ef835ba
Jul 21 12:34:28.403407 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:28.403364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz" event={"ID":"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d","Type":"ContainerStarted","Data":"a4b4e4c882b74213f06a219ae2f0450e6ff27ebe168274afcf67ab894ef835ba"}
Jul 21 12:34:29.422692 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.422654    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"]
Jul 21 12:34:29.426093 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.426072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:29.430109 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.430086    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7mqtf\""
Jul 21 12:34:29.451790 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.451763    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"]
Jul 21 12:34:29.506523 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.506491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7fnk\" (UniqueName: \"kubernetes.io/projected/93da7e2f-1218-4f75-a55c-c38d299ea817-kube-api-access-l7fnk\") pod \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" (UID: \"93da7e2f-1218-4f75-a55c-c38d299ea817\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:29.606944 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.606901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7fnk\" (UniqueName: \"kubernetes.io/projected/93da7e2f-1218-4f75-a55c-c38d299ea817-kube-api-access-l7fnk\") pod \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" (UID: \"93da7e2f-1218-4f75-a55c-c38d299ea817\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:29.618161 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.618130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7fnk\" (UniqueName: \"kubernetes.io/projected/93da7e2f-1218-4f75-a55c-c38d299ea817-kube-api-access-l7fnk\") pod \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" (UID: \"93da7e2f-1218-4f75-a55c-c38d299ea817\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:29.736940 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.736907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:29.869482 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:29.869423    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"]
Jul 21 12:34:29.872264 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:34:29.872233    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93da7e2f_1218_4f75_a55c_c38d299ea817.slice/crio-dac81ce987a384a617e9b825f65771bc9aa85d1ff9b4ee4dd3251a44b56fd520 WatchSource:0}: Error finding container dac81ce987a384a617e9b825f65771bc9aa85d1ff9b4ee4dd3251a44b56fd520: Status 404 returned error can't find the container with id dac81ce987a384a617e9b825f65771bc9aa85d1ff9b4ee4dd3251a44b56fd520
Jul 21 12:34:30.414024 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:30.413989    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q" event={"ID":"93da7e2f-1218-4f75-a55c-c38d299ea817","Type":"ContainerStarted","Data":"dac81ce987a384a617e9b825f65771bc9aa85d1ff9b4ee4dd3251a44b56fd520"}
Jul 21 12:34:33.431118 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:33.431082    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q" event={"ID":"93da7e2f-1218-4f75-a55c-c38d299ea817","Type":"ContainerStarted","Data":"ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877"}
Jul 21 12:34:33.431546 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:33.431264    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:33.432835 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:33.432810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz" event={"ID":"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d","Type":"ContainerStarted","Data":"a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf"}
Jul 21 12:34:33.432990 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:33.432946    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:33.451952 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:33.451914    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q" podStartSLOduration=1.607604572 podStartE2EDuration="4.451902358s" totalImagesPullingTime="2.844297786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:33.451593847 +0000 UTC m=+411.236608826" watchObservedRunningTime="2026-07-21 12:34:33.451902358 +0000 UTC m=+411.236917323"
Jul 21 12:34:33.481018 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:33.480970    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz" podStartSLOduration=1.302185774 podStartE2EDuration="6.48094033s" totalImagesPullingTime="5.178754556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:33.478740525 +0000 UTC m=+411.263755505" watchObservedRunningTime="2026-07-21 12:34:33.48094033 +0000 UTC m=+411.265955295"
Jul 21 12:34:35.500643 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.500616    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"]
Jul 21 12:34:35.503743 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.503724    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.506283 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.506254    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 21 12:34:35.506391 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.506289    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ckjqb\""
Jul 21 12:34:35.506563 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.506545    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 21 12:34:35.515975 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.514556    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"]
Jul 21 12:34:35.558458 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.558437    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctb5d\" (UniqueName: \"kubernetes.io/projected/d956aff0-9f68-49d3-be2e-58ee02f99848-kube-api-access-ctb5d\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.558563 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.558467    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d956aff0-9f68-49d3-be2e-58ee02f99848-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.558563 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.558491    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d956aff0-9f68-49d3-be2e-58ee02f99848-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.659588 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.659551    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctb5d\" (UniqueName: \"kubernetes.io/projected/d956aff0-9f68-49d3-be2e-58ee02f99848-kube-api-access-ctb5d\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.659777 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.659602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d956aff0-9f68-49d3-be2e-58ee02f99848-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.659777 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.659642    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d956aff0-9f68-49d3-be2e-58ee02f99848-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.659887 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:35.659805    2577 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 21 12:34:35.659887 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:35.659883    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d956aff0-9f68-49d3-be2e-58ee02f99848-plugin-serving-cert podName:d956aff0-9f68-49d3-be2e-58ee02f99848 nodeName:}" failed. No retries permitted until 2026-07-21 12:34:36.159859373 +0000 UTC m=+413.944874328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/d956aff0-9f68-49d3-be2e-58ee02f99848-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-t2dqs" (UID: "d956aff0-9f68-49d3-be2e-58ee02f99848") : secret "plugin-serving-cert" not found
Jul 21 12:34:35.660517 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.660488    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d956aff0-9f68-49d3-be2e-58ee02f99848-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:35.672364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:35.672337    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctb5d\" (UniqueName: \"kubernetes.io/projected/d956aff0-9f68-49d3-be2e-58ee02f99848-kube-api-access-ctb5d\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:36.164981 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:36.164920    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d956aff0-9f68-49d3-be2e-58ee02f99848-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:36.167301 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:36.167282    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d956aff0-9f68-49d3-be2e-58ee02f99848-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-t2dqs\" (UID: \"d956aff0-9f68-49d3-be2e-58ee02f99848\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:36.413600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:36.413549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"
Jul 21 12:34:36.548486 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:36.548369    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs"]
Jul 21 12:34:36.550707 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:34:36.550678    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd956aff0_9f68_49d3_be2e_58ee02f99848.slice/crio-d7e73b1c3ed35388d4902a3547a3ad997d99b71117b3591abaab32a897e1ab4e WatchSource:0}: Error finding container d7e73b1c3ed35388d4902a3547a3ad997d99b71117b3591abaab32a897e1ab4e: Status 404 returned error can't find the container with id d7e73b1c3ed35388d4902a3547a3ad997d99b71117b3591abaab32a897e1ab4e
Jul 21 12:34:37.399232 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:37.399194    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-x8gz5"
Jul 21 12:34:37.455209 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:37.455132    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs" event={"ID":"d956aff0-9f68-49d3-be2e-58ee02f99848","Type":"ContainerStarted","Data":"d7e73b1c3ed35388d4902a3547a3ad997d99b71117b3591abaab32a897e1ab4e"}
Jul 21 12:34:44.440241 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:44.440208    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:44.440690 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:44.440301    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:46.401494 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.401458    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"]
Jul 21 12:34:46.401951 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.401683    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" containerName="manager" containerID="cri-o://a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf" gracePeriod=2
Jul 21 12:34:46.408541 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.408498    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"]
Jul 21 12:34:46.411812 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:46.411731    2577 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-685lz\": pods \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:46.448632 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.448598    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"]
Jul 21 12:34:46.449122 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.449101    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" containerName="manager"
Jul 21 12:34:46.449122 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.449124    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" containerName="manager"
Jul 21 12:34:46.449281 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.449203    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" containerName="manager"
Jul 21 12:34:46.454321 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.454298    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"]
Jul 21 12:34:46.454456 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.454432    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.454618 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.454573    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" containerName="manager" containerID="cri-o://ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877" gracePeriod=2
Jul 21 12:34:46.462935 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:46.462224    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:46.462935 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.462725    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"]
Jul 21 12:34:46.474869 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.474842    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"]
Jul 21 12:34:46.499010 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:46.498797    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:46.499450 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.499212    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"]
Jul 21 12:34:46.500364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.499944    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" containerName="manager"
Jul 21 12:34:46.500364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.499990    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" containerName="manager"
Jul 21 12:34:46.500364 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.500105    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" containerName="manager"
Jul 21 12:34:46.501173 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:46.501136    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:46.505037 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.504992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"
Jul 21 12:34:46.507557 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:46.507518    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:46.516991 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.516950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"]
Jul 21 12:34:46.535165 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:46.535123    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:46.563705 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.563679    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb1203d6-dee2-4718-81fc-38f508fd5229-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tmrt7\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.563829 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.563744    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjf7r\" (UniqueName: \"kubernetes.io/projected/bb1203d6-dee2-4718-81fc-38f508fd5229-kube-api-access-xjf7r\") pod \"kuadrant-operator-controller-manager-86878c769f-tmrt7\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.664872 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.664775    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb1203d6-dee2-4718-81fc-38f508fd5229-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tmrt7\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.665045 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.664940    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjf7r\" (UniqueName: \"kubernetes.io/projected/bb1203d6-dee2-4718-81fc-38f508fd5229-kube-api-access-xjf7r\") pod \"kuadrant-operator-controller-manager-86878c769f-tmrt7\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.665045 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.664988    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ljnc\" (UniqueName: \"kubernetes.io/projected/a730b152-a7cf-47c2-a015-5d60630ed037-kube-api-access-8ljnc\") pod \"limitador-operator-controller-manager-859d7b78bd-qnpdj\" (UID: \"a730b152-a7cf-47c2-a015-5d60630ed037\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"
Jul 21 12:34:46.665228 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.665205    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb1203d6-dee2-4718-81fc-38f508fd5229-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-tmrt7\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.676918 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.676890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjf7r\" (UniqueName: \"kubernetes.io/projected/bb1203d6-dee2-4718-81fc-38f508fd5229-kube-api-access-xjf7r\") pod \"kuadrant-operator-controller-manager-86878c769f-tmrt7\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.766446 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.766394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ljnc\" (UniqueName: \"kubernetes.io/projected/a730b152-a7cf-47c2-a015-5d60630ed037-kube-api-access-8ljnc\") pod \"limitador-operator-controller-manager-859d7b78bd-qnpdj\" (UID: \"a730b152-a7cf-47c2-a015-5d60630ed037\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"
Jul 21 12:34:46.781796 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.781768    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ljnc\" (UniqueName: \"kubernetes.io/projected/a730b152-a7cf-47c2-a015-5d60630ed037-kube-api-access-8ljnc\") pod \"limitador-operator-controller-manager-859d7b78bd-qnpdj\" (UID: \"a730b152-a7cf-47c2-a015-5d60630ed037\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"
Jul 21 12:34:46.846286 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.846247    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:46.854615 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:46.854587    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"
Jul 21 12:34:58.098796 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.098725    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"]
Jul 21 12:34:58.112147 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.112108    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:58.113738 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.113714    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:58.115315 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.115290    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:58.117655 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.117616    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:58.119585 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.119555    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:58.142353 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.142332    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"]
Jul 21 12:34:58.147622 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:34:58.147599    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda730b152_a7cf_47c2_a015_5d60630ed037.slice/crio-d16d5ca8cff1be615efb3f56a96db2cedf981a076b5264280fac1ddfdc222c60 WatchSource:0}: Error finding container d16d5ca8cff1be615efb3f56a96db2cedf981a076b5264280fac1ddfdc222c60: Status 404 returned error can't find the container with id d16d5ca8cff1be615efb3f56a96db2cedf981a076b5264280fac1ddfdc222c60
Jul 21 12:34:58.272502 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.272469    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7fnk\" (UniqueName: \"kubernetes.io/projected/93da7e2f-1218-4f75-a55c-c38d299ea817-kube-api-access-l7fnk\") pod \"93da7e2f-1218-4f75-a55c-c38d299ea817\" (UID: \"93da7e2f-1218-4f75-a55c-c38d299ea817\") "
Jul 21 12:34:58.272673 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.272592    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-extensions-socket-volume\") pod \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") "
Jul 21 12:34:58.272673 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.272619    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ts6b\" (UniqueName: \"kubernetes.io/projected/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-kube-api-access-4ts6b\") pod \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\" (UID: \"9a1cc992-38e6-442b-9ba0-c394c9ab8c4d\") "
Jul 21 12:34:58.272999 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.272948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" (UID: "9a1cc992-38e6-442b-9ba0-c394c9ab8c4d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:34:58.274491 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.274472    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93da7e2f-1218-4f75-a55c-c38d299ea817-kube-api-access-l7fnk" (OuterVolumeSpecName: "kube-api-access-l7fnk") pod "93da7e2f-1218-4f75-a55c-c38d299ea817" (UID: "93da7e2f-1218-4f75-a55c-c38d299ea817"). InnerVolumeSpecName "kube-api-access-l7fnk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:34:58.274558 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.274525    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-kube-api-access-4ts6b" (OuterVolumeSpecName: "kube-api-access-4ts6b") pod "9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" (UID: "9a1cc992-38e6-442b-9ba0-c394c9ab8c4d"). InnerVolumeSpecName "kube-api-access-4ts6b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:34:58.373375 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.373345    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-extensions-socket-volume\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:34:58.373375 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.373370    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ts6b\" (UniqueName: \"kubernetes.io/projected/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d-kube-api-access-4ts6b\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:34:58.373375 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.373379    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7fnk\" (UniqueName: \"kubernetes.io/projected/93da7e2f-1218-4f75-a55c-c38d299ea817-kube-api-access-l7fnk\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:34:58.564848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.564806    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs" event={"ID":"d956aff0-9f68-49d3-be2e-58ee02f99848","Type":"ContainerStarted","Data":"6a25dcc1d1d92344cc78160a9e1ce1f3c84c5171c6c2099ac890797527fbaa39"}
Jul 21 12:34:58.566506 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.566479    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7" event={"ID":"bb1203d6-dee2-4718-81fc-38f508fd5229","Type":"ContainerStarted","Data":"f667cc2a1cda40f111a5913ffac261b2cd996db3d19c74b3172154d2e6d5f902"}
Jul 21 12:34:58.566618 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.566526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7" event={"ID":"bb1203d6-dee2-4718-81fc-38f508fd5229","Type":"ContainerStarted","Data":"008e12fbbb9bfcccff73f23c300d7a7250519764c8f7fb28c8697b15184e790e"}
Jul 21 12:34:58.566618 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.566574    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:34:58.567695 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.567676    2577 generic.go:356] "Generic (PLEG): container finished" podID="93da7e2f-1218-4f75-a55c-c38d299ea817" containerID="ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877" exitCode=0
Jul 21 12:34:58.567758 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.567707    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:58.567802 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.567766    2577 scope.go:117] "RemoveContainer" containerID="ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877"
Jul 21 12:34:58.569033 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.569014    2577 generic.go:356] "Generic (PLEG): container finished" podID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" containerID="a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf" exitCode=0
Jul 21 12:34:58.569309 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.569055    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:58.570695 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.570673    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj" event={"ID":"a730b152-a7cf-47c2-a015-5d60630ed037","Type":"ContainerStarted","Data":"b944c9f7dece7213f3e726b91687f6d1043a38393d5954a605e051880aa2047f"}
Jul 21 12:34:58.570775 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.570728    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"
Jul 21 12:34:58.570775 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.570752    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj" event={"ID":"a730b152-a7cf-47c2-a015-5d60630ed037","Type":"ContainerStarted","Data":"d16d5ca8cff1be615efb3f56a96db2cedf981a076b5264280fac1ddfdc222c60"}
Jul 21 12:34:58.576466 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.576449    2577 scope.go:117] "RemoveContainer" containerID="ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877"
Jul 21 12:34:58.576705 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.576685    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877\": container with ID starting with ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877 not found: ID does not exist" containerID="ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877"
Jul 21 12:34:58.576766 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.576711    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877"} err="failed to get container status \"ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877\": rpc error: code = NotFound desc = could not find container \"ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877\": container with ID starting with ee9d59f0bcb6032f3bcbb2224435d3e0692b8affed464ade49d8e9f136b2c877 not found: ID does not exist"
Jul 21 12:34:58.576766 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.576726    2577 scope.go:117] "RemoveContainer" containerID="a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf"
Jul 21 12:34:58.584502 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.584484    2577 scope.go:117] "RemoveContainer" containerID="a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf"
Jul 21 12:34:58.584744 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.584727    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf\": container with ID starting with a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf not found: ID does not exist" containerID="a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf"
Jul 21 12:34:58.584785 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.584751    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf"} err="failed to get container status \"a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf\": rpc error: code = NotFound desc = could not find container \"a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf\": container with ID starting with a4b7954159f3710e149921790cc93893a7b861fed66123dce8678cc1d4c5ccaf not found: ID does not exist"
Jul 21 12:34:58.589054 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.589017    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-t2dqs" podStartSLOduration=2.139755612 podStartE2EDuration="23.58900505s" totalImagesPullingTime="21.449249438s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:58.587750021 +0000 UTC m=+436.372764987" watchObservedRunningTime="2026-07-21 12:34:58.58900505 +0000 UTC m=+436.374020044"
Jul 21 12:34:58.589798 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.589776    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:58.591718 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.591696    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:58.621272 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.621222    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7" podStartSLOduration=12.621201618 podStartE2EDuration="12.621201618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:58.620476106 +0000 UTC m=+436.405491073" watchObservedRunningTime="2026-07-21 12:34:58.621201618 +0000 UTC m=+436.406216583"
Jul 21 12:34:58.622584 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.622553    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-kkm5q\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-kkm5q"
Jul 21 12:34:58.647662 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.647602    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj" podStartSLOduration=12.647584334 podStartE2EDuration="12.647584334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:34:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:34:58.64587664 +0000 UTC m=+436.430891626" watchObservedRunningTime="2026-07-21 12:34:58.647584334 +0000 UTC m=+436.432599304"
Jul 21 12:34:58.648046 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:34:58.648022    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-685lz\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-685lz"
Jul 21 12:34:58.832681 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.832600    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93da7e2f-1218-4f75-a55c-c38d299ea817" path="/var/lib/kubelet/pods/93da7e2f-1218-4f75-a55c-c38d299ea817/volumes"
Jul 21 12:34:58.833101 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:34:58.833083    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a1cc992-38e6-442b-9ba0-c394c9ab8c4d" path="/var/lib/kubelet/pods/9a1cc992-38e6-442b-9ba0-c394c9ab8c4d/volumes"
Jul 21 12:35:09.578669 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:09.578582    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-qnpdj"
Jul 21 12:35:09.579125 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:09.578671    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:35:32.156352 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.156321    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-vnmsc"]
Jul 21 12:35:32.184351 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.184322    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vnmsc"]
Jul 21 12:35:32.184490 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.184409    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vnmsc"
Jul 21 12:35:32.186847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.186823    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6rhvw\""
Jul 21 12:35:32.215200 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.215177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ndtd\" (UniqueName: \"kubernetes.io/projected/e8c784ff-fb1a-4495-9cd7-5d9e4601d10e-kube-api-access-4ndtd\") pod \"authorino-58cd955675-vnmsc\" (UID: \"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e\") " pod="kuadrant-system/authorino-58cd955675-vnmsc"
Jul 21 12:35:32.316417 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.316386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ndtd\" (UniqueName: \"kubernetes.io/projected/e8c784ff-fb1a-4495-9cd7-5d9e4601d10e-kube-api-access-4ndtd\") pod \"authorino-58cd955675-vnmsc\" (UID: \"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e\") " pod="kuadrant-system/authorino-58cd955675-vnmsc"
Jul 21 12:35:32.326272 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.326241    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ndtd\" (UniqueName: \"kubernetes.io/projected/e8c784ff-fb1a-4495-9cd7-5d9e4601d10e-kube-api-access-4ndtd\") pod \"authorino-58cd955675-vnmsc\" (UID: \"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e\") " pod="kuadrant-system/authorino-58cd955675-vnmsc"
Jul 21 12:35:32.493303 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.493277    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vnmsc"
Jul 21 12:35:32.619661 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.619632    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-vnmsc"]
Jul 21 12:35:32.623188 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:35:32.623157    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode8c784ff_fb1a_4495_9cd7_5d9e4601d10e.slice/crio-54c9aa48af4e6d7071084e64d2dde4a86f27d78552a196e644df6f0957bff321 WatchSource:0}: Error finding container 54c9aa48af4e6d7071084e64d2dde4a86f27d78552a196e644df6f0957bff321: Status 404 returned error can't find the container with id 54c9aa48af4e6d7071084e64d2dde4a86f27d78552a196e644df6f0957bff321
Jul 21 12:35:32.727878 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:32.727844    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vnmsc" event={"ID":"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e","Type":"ContainerStarted","Data":"54c9aa48af4e6d7071084e64d2dde4a86f27d78552a196e644df6f0957bff321"}
Jul 21 12:35:35.750903 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:35.750864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vnmsc" event={"ID":"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e","Type":"ContainerStarted","Data":"f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17"}
Jul 21 12:35:37.386017 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:37.385936    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-vnmsc" podStartSLOduration=2.409217292 podStartE2EDuration="5.385912783s" totalImagesPullingTime="2.976695491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:35:35.76482527 +0000 UTC m=+473.549840238" watchObservedRunningTime="2026-07-21 12:35:37.385912783 +0000 UTC m=+475.170927751"
Jul 21 12:35:37.387662 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:37.387634    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vnmsc"]
Jul 21 12:35:37.759621 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:37.759582    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-vnmsc" podUID="e8c784ff-fb1a-4495-9cd7-5d9e4601d10e" containerName="authorino" containerID="cri-o://f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17" gracePeriod=30
Jul 21 12:35:37.998490 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:37.998469    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vnmsc"
Jul 21 12:35:38.068561 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.068476    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ndtd\" (UniqueName: \"kubernetes.io/projected/e8c784ff-fb1a-4495-9cd7-5d9e4601d10e-kube-api-access-4ndtd\") pod \"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e\" (UID: \"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e\") "
Jul 21 12:35:38.070521 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.070486    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8c784ff-fb1a-4495-9cd7-5d9e4601d10e-kube-api-access-4ndtd" (OuterVolumeSpecName: "kube-api-access-4ndtd") pod "e8c784ff-fb1a-4495-9cd7-5d9e4601d10e" (UID: "e8c784ff-fb1a-4495-9cd7-5d9e4601d10e"). InnerVolumeSpecName "kube-api-access-4ndtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:35:38.170022 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.169985    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ndtd\" (UniqueName: \"kubernetes.io/projected/e8c784ff-fb1a-4495-9cd7-5d9e4601d10e-kube-api-access-4ndtd\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:35:38.764497 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.764468    2577 generic.go:356] "Generic (PLEG): container finished" podID="e8c784ff-fb1a-4495-9cd7-5d9e4601d10e" containerID="f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17" exitCode=0
Jul 21 12:35:38.765001 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.764517    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-vnmsc"
Jul 21 12:35:38.765001 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.764552    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vnmsc" event={"ID":"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e","Type":"ContainerDied","Data":"f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17"}
Jul 21 12:35:38.765001 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.764595    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-vnmsc" event={"ID":"e8c784ff-fb1a-4495-9cd7-5d9e4601d10e","Type":"ContainerDied","Data":"54c9aa48af4e6d7071084e64d2dde4a86f27d78552a196e644df6f0957bff321"}
Jul 21 12:35:38.765001 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.764614    2577 scope.go:117] "RemoveContainer" containerID="f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17"
Jul 21 12:35:38.773369 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.773353    2577 scope.go:117] "RemoveContainer" containerID="f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17"
Jul 21 12:35:38.773640 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:35:38.773622    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17\": container with ID starting with f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17 not found: ID does not exist" containerID="f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17"
Jul 21 12:35:38.773706 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.773646    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17"} err="failed to get container status \"f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17\": rpc error: code = NotFound desc = could not find container \"f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17\": container with ID starting with f4463fb11859a436b4414843c19e68e8bdeda7c0fb18b54b5bd1b25016f56d17 not found: ID does not exist"
Jul 21 12:35:38.784252 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.784231    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-vnmsc"]
Jul 21 12:35:38.788157 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.788139    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-vnmsc"]
Jul 21 12:35:38.831534 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:35:38.831512    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8c784ff-fb1a-4495-9cd7-5d9e4601d10e" path="/var/lib/kubelet/pods/e8c784ff-fb1a-4495-9cd7-5d9e4601d10e/volumes"
Jul 21 12:36:04.967767 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:04.967735    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tc6rr"]
Jul 21 12:36:04.968243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:04.968053    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8c784ff-fb1a-4495-9cd7-5d9e4601d10e" containerName="authorino"
Jul 21 12:36:04.968243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:04.968064    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8c784ff-fb1a-4495-9cd7-5d9e4601d10e" containerName="authorino"
Jul 21 12:36:04.968243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:04.968127    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8c784ff-fb1a-4495-9cd7-5d9e4601d10e" containerName="authorino"
Jul 21 12:36:04.974506 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:04.974476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tc6rr"
Jul 21 12:36:04.977681 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:04.977661    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6rhvw\""
Jul 21 12:36:04.977817 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:04.977758    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tc6rr"]
Jul 21 12:36:05.077513 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.077481    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnhb4\" (UniqueName: \"kubernetes.io/projected/9d3b7279-b646-4f36-96f3-61f68e8aab25-kube-api-access-vnhb4\") pod \"authorino-5d4b67c446-tc6rr\" (UID: \"9d3b7279-b646-4f36-96f3-61f68e8aab25\") " pod="kuadrant-system/authorino-5d4b67c446-tc6rr"
Jul 21 12:36:05.178150 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.178115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnhb4\" (UniqueName: \"kubernetes.io/projected/9d3b7279-b646-4f36-96f3-61f68e8aab25-kube-api-access-vnhb4\") pod \"authorino-5d4b67c446-tc6rr\" (UID: \"9d3b7279-b646-4f36-96f3-61f68e8aab25\") " pod="kuadrant-system/authorino-5d4b67c446-tc6rr"
Jul 21 12:36:05.186338 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.186313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnhb4\" (UniqueName: \"kubernetes.io/projected/9d3b7279-b646-4f36-96f3-61f68e8aab25-kube-api-access-vnhb4\") pod \"authorino-5d4b67c446-tc6rr\" (UID: \"9d3b7279-b646-4f36-96f3-61f68e8aab25\") " pod="kuadrant-system/authorino-5d4b67c446-tc6rr"
Jul 21 12:36:05.212446 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.212420    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tc6rr"]
Jul 21 12:36:05.212648 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.212637    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tc6rr"
Jul 21 12:36:05.237941 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.237917    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7d485bdfdd-wj7n8"]
Jul 21 12:36:05.241544 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.241478    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8"
Jul 21 12:36:05.248073 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.248048    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7d485bdfdd-wj7n8"]
Jul 21 12:36:05.350169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.350125    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tc6rr"]
Jul 21 12:36:05.352750 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:36:05.352723    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d3b7279_b646_4f36_96f3_61f68e8aab25.slice/crio-baf3b98c9afa58887f6f0ccdf42bef1ccd8b4b18326f8c72b8184a67d8fcc25e WatchSource:0}: Error finding container baf3b98c9afa58887f6f0ccdf42bef1ccd8b4b18326f8c72b8184a67d8fcc25e: Status 404 returned error can't find the container with id baf3b98c9afa58887f6f0ccdf42bef1ccd8b4b18326f8c72b8184a67d8fcc25e
Jul 21 12:36:05.379842 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.379821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2nqt\" (UniqueName: \"kubernetes.io/projected/c003f537-f826-44e9-91f6-148f78ffb946-kube-api-access-q2nqt\") pod \"authorino-7d485bdfdd-wj7n8\" (UID: \"c003f537-f826-44e9-91f6-148f78ffb946\") " pod="kuadrant-system/authorino-7d485bdfdd-wj7n8"
Jul 21 12:36:05.480381 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.480320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2nqt\" (UniqueName: \"kubernetes.io/projected/c003f537-f826-44e9-91f6-148f78ffb946-kube-api-access-q2nqt\") pod \"authorino-7d485bdfdd-wj7n8\" (UID: \"c003f537-f826-44e9-91f6-148f78ffb946\") " pod="kuadrant-system/authorino-7d485bdfdd-wj7n8"
Jul 21 12:36:05.488715 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.488686    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2nqt\" (UniqueName: \"kubernetes.io/projected/c003f537-f826-44e9-91f6-148f78ffb946-kube-api-access-q2nqt\") pod \"authorino-7d485bdfdd-wj7n8\" (UID: \"c003f537-f826-44e9-91f6-148f78ffb946\") " pod="kuadrant-system/authorino-7d485bdfdd-wj7n8"
Jul 21 12:36:05.525300 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.525277    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d485bdfdd-wj7n8"]
Jul 21 12:36:05.525479 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.525468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8"
Jul 21 12:36:05.551284 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.551259    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-cb57694d8-rj9b2"]
Jul 21 12:36:05.554381 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.554364    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.556556 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.556540    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 21 12:36:05.560864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.560841    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb57694d8-rj9b2"]
Jul 21 12:36:05.650334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.650310    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d485bdfdd-wj7n8"]
Jul 21 12:36:05.652711 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:36:05.652682    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc003f537_f826_44e9_91f6_148f78ffb946.slice/crio-576c46a23a8927ab5e507c9604e24275e870a97db8d2edc0d515875f8ce8b9f9 WatchSource:0}: Error finding container 576c46a23a8927ab5e507c9604e24275e870a97db8d2edc0d515875f8ce8b9f9: Status 404 returned error can't find the container with id 576c46a23a8927ab5e507c9604e24275e870a97db8d2edc0d515875f8ce8b9f9
Jul 21 12:36:05.681862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.681840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e570331f-3035-4c28-a599-769955d89d78-tls-cert\") pod \"authorino-cb57694d8-rj9b2\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") " pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.681983 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.681903    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96fkp\" (UniqueName: \"kubernetes.io/projected/e570331f-3035-4c28-a599-769955d89d78-kube-api-access-96fkp\") pod \"authorino-cb57694d8-rj9b2\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") " pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.783183 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.783159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96fkp\" (UniqueName: \"kubernetes.io/projected/e570331f-3035-4c28-a599-769955d89d78-kube-api-access-96fkp\") pod \"authorino-cb57694d8-rj9b2\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") " pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.783294 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.783211    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e570331f-3035-4c28-a599-769955d89d78-tls-cert\") pod \"authorino-cb57694d8-rj9b2\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") " pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.785448 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.785424    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e570331f-3035-4c28-a599-769955d89d78-tls-cert\") pod \"authorino-cb57694d8-rj9b2\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") " pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.790487 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.790466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96fkp\" (UniqueName: \"kubernetes.io/projected/e570331f-3035-4c28-a599-769955d89d78-kube-api-access-96fkp\") pod \"authorino-cb57694d8-rj9b2\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") " pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.865542 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.865521    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:36:05.872006 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.871981    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8" event={"ID":"c003f537-f826-44e9-91f6-148f78ffb946","Type":"ContainerStarted","Data":"576c46a23a8927ab5e507c9604e24275e870a97db8d2edc0d515875f8ce8b9f9"}
Jul 21 12:36:05.873166 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:05.873143    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tc6rr" event={"ID":"9d3b7279-b646-4f36-96f3-61f68e8aab25","Type":"ContainerStarted","Data":"baf3b98c9afa58887f6f0ccdf42bef1ccd8b4b18326f8c72b8184a67d8fcc25e"}
Jul 21 12:36:06.006225 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.006058    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-cb57694d8-rj9b2"]
Jul 21 12:36:06.008853 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:36:06.008818    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode570331f_3035_4c28_a599_769955d89d78.slice/crio-2758751b0b0b427a27c9bbf07d088dfbf14c2d0c4f27908cffb27995ceba3c94 WatchSource:0}: Error finding container 2758751b0b0b427a27c9bbf07d088dfbf14c2d0c4f27908cffb27995ceba3c94: Status 404 returned error can't find the container with id 2758751b0b0b427a27c9bbf07d088dfbf14c2d0c4f27908cffb27995ceba3c94
Jul 21 12:36:06.878058 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.878018    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8" event={"ID":"c003f537-f826-44e9-91f6-148f78ffb946","Type":"ContainerStarted","Data":"156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905"}
Jul 21 12:36:06.878058 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.878049    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8" podUID="c003f537-f826-44e9-91f6-148f78ffb946" containerName="authorino" containerID="cri-o://156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905" gracePeriod=30
Jul 21 12:36:06.879539 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.879513    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-tc6rr" podUID="9d3b7279-b646-4f36-96f3-61f68e8aab25" containerName="authorino" containerID="cri-o://7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419" gracePeriod=30
Jul 21 12:36:06.879665 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.879519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tc6rr" event={"ID":"9d3b7279-b646-4f36-96f3-61f68e8aab25","Type":"ContainerStarted","Data":"7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419"}
Jul 21 12:36:06.881088 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.881061    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb57694d8-rj9b2" event={"ID":"e570331f-3035-4c28-a599-769955d89d78","Type":"ContainerStarted","Data":"730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb"}
Jul 21 12:36:06.881197 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.881099    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb57694d8-rj9b2" event={"ID":"e570331f-3035-4c28-a599-769955d89d78","Type":"ContainerStarted","Data":"2758751b0b0b427a27c9bbf07d088dfbf14c2d0c4f27908cffb27995ceba3c94"}
Jul 21 12:36:06.892140 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.892094    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8" podStartSLOduration=1.488175054 podStartE2EDuration="1.89207825s" totalImagesPullingTime="403.903196ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:36:06.891737265 +0000 UTC m=+504.676752232" watchObservedRunningTime="2026-07-21 12:36:06.89207825 +0000 UTC m=+504.677093215"
Jul 21 12:36:06.905691 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.905649    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-tc6rr" podStartSLOduration=2.376264043 podStartE2EDuration="2.905637294s" totalImagesPullingTime="529.373251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:36:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:36:06.904735252 +0000 UTC m=+504.689750217" watchObservedRunningTime="2026-07-21 12:36:06.905637294 +0000 UTC m=+504.690652266"
Jul 21 12:36:06.919425 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:06.919386    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-cb57694d8-rj9b2" podStartSLOduration=1.498255363 podStartE2EDuration="1.919374123s" totalImagesPullingTime="421.11876ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:36:06.917924912 +0000 UTC m=+504.702939876" watchObservedRunningTime="2026-07-21 12:36:06.919374123 +0000 UTC m=+504.704389087"
Jul 21 12:36:07.122708 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.122684    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8"
Jul 21 12:36:07.129291 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.129246    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tc6rr"
Jul 21 12:36:07.194768 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.194744    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2nqt\" (UniqueName: \"kubernetes.io/projected/c003f537-f826-44e9-91f6-148f78ffb946-kube-api-access-q2nqt\") pod \"c003f537-f826-44e9-91f6-148f78ffb946\" (UID: \"c003f537-f826-44e9-91f6-148f78ffb946\") "
Jul 21 12:36:07.196809 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.196780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c003f537-f826-44e9-91f6-148f78ffb946-kube-api-access-q2nqt" (OuterVolumeSpecName: "kube-api-access-q2nqt") pod "c003f537-f826-44e9-91f6-148f78ffb946" (UID: "c003f537-f826-44e9-91f6-148f78ffb946"). InnerVolumeSpecName "kube-api-access-q2nqt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:36:07.295125 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.295104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnhb4\" (UniqueName: \"kubernetes.io/projected/9d3b7279-b646-4f36-96f3-61f68e8aab25-kube-api-access-vnhb4\") pod \"9d3b7279-b646-4f36-96f3-61f68e8aab25\" (UID: \"9d3b7279-b646-4f36-96f3-61f68e8aab25\") "
Jul 21 12:36:07.295334 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.295318    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2nqt\" (UniqueName: \"kubernetes.io/projected/c003f537-f826-44e9-91f6-148f78ffb946-kube-api-access-q2nqt\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:36:07.297076 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.297051    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d3b7279-b646-4f36-96f3-61f68e8aab25-kube-api-access-vnhb4" (OuterVolumeSpecName: "kube-api-access-vnhb4") pod "9d3b7279-b646-4f36-96f3-61f68e8aab25" (UID: "9d3b7279-b646-4f36-96f3-61f68e8aab25"). InnerVolumeSpecName "kube-api-access-vnhb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:36:07.396649 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.396589    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnhb4\" (UniqueName: \"kubernetes.io/projected/9d3b7279-b646-4f36-96f3-61f68e8aab25-kube-api-access-vnhb4\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:36:07.886707 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.886673    2577 generic.go:356] "Generic (PLEG): container finished" podID="c003f537-f826-44e9-91f6-148f78ffb946" containerID="156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905" exitCode=0
Jul 21 12:36:07.886850 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.886720    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8"
Jul 21 12:36:07.886850 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.886759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8" event={"ID":"c003f537-f826-44e9-91f6-148f78ffb946","Type":"ContainerDied","Data":"156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905"}
Jul 21 12:36:07.886850 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.886798    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7d485bdfdd-wj7n8" event={"ID":"c003f537-f826-44e9-91f6-148f78ffb946","Type":"ContainerDied","Data":"576c46a23a8927ab5e507c9604e24275e870a97db8d2edc0d515875f8ce8b9f9"}
Jul 21 12:36:07.886850 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.886818    2577 scope.go:117] "RemoveContainer" containerID="156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905"
Jul 21 12:36:07.887990 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.887930    2577 generic.go:356] "Generic (PLEG): container finished" podID="9d3b7279-b646-4f36-96f3-61f68e8aab25" containerID="7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419" exitCode=0
Jul 21 12:36:07.888067 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.887984    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tc6rr" event={"ID":"9d3b7279-b646-4f36-96f3-61f68e8aab25","Type":"ContainerDied","Data":"7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419"}
Jul 21 12:36:07.888067 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.887999    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tc6rr"
Jul 21 12:36:07.888067 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.888017    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-tc6rr" event={"ID":"9d3b7279-b646-4f36-96f3-61f68e8aab25","Type":"ContainerDied","Data":"baf3b98c9afa58887f6f0ccdf42bef1ccd8b4b18326f8c72b8184a67d8fcc25e"}
Jul 21 12:36:07.896401 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.896308    2577 scope.go:117] "RemoveContainer" containerID="156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905"
Jul 21 12:36:07.896657 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:36:07.896639    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905\": container with ID starting with 156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905 not found: ID does not exist" containerID="156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905"
Jul 21 12:36:07.896744 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.896666    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905"} err="failed to get container status \"156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905\": rpc error: code = NotFound desc = could not find container \"156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905\": container with ID starting with 156f5fdb79944b3ccd3502bd11fc48b7db79a78d5e903f33ecd0f3d4052ec905 not found: ID does not exist"
Jul 21 12:36:07.896744 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.896686    2577 scope.go:117] "RemoveContainer" containerID="7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419"
Jul 21 12:36:07.904043 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.904028    2577 scope.go:117] "RemoveContainer" containerID="7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419"
Jul 21 12:36:07.904273 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:36:07.904256    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419\": container with ID starting with 7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419 not found: ID does not exist" containerID="7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419"
Jul 21 12:36:07.904319 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.904279    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419"} err="failed to get container status \"7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419\": rpc error: code = NotFound desc = could not find container \"7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419\": container with ID starting with 7eb2ae3511d567570555a91eee78edd763dcf852ee93978187c06bcd425ce419 not found: ID does not exist"
Jul 21 12:36:07.910529 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.910509    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7d485bdfdd-wj7n8"]
Jul 21 12:36:07.914207 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.914185    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7d485bdfdd-wj7n8"]
Jul 21 12:36:07.924098 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.924065    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tc6rr"]
Jul 21 12:36:07.927761 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:07.927743    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tc6rr"]
Jul 21 12:36:08.831969 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:08.831932    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d3b7279-b646-4f36-96f3-61f68e8aab25" path="/var/lib/kubelet/pods/9d3b7279-b646-4f36-96f3-61f68e8aab25/volumes"
Jul 21 12:36:08.832321 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:36:08.832240    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c003f537-f826-44e9-91f6-148f78ffb946" path="/var/lib/kubelet/pods/c003f537-f826-44e9-91f6-148f78ffb946/volumes"
Jul 21 12:37:40.091864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.091834    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"]
Jul 21 12:37:40.092310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.092171    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d3b7279-b646-4f36-96f3-61f68e8aab25" containerName="authorino"
Jul 21 12:37:40.092310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.092182    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d3b7279-b646-4f36-96f3-61f68e8aab25" containerName="authorino"
Jul 21 12:37:40.092310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.092190    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c003f537-f826-44e9-91f6-148f78ffb946" containerName="authorino"
Jul 21 12:37:40.092310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.092196    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c003f537-f826-44e9-91f6-148f78ffb946" containerName="authorino"
Jul 21 12:37:40.092310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.092246    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d3b7279-b646-4f36-96f3-61f68e8aab25" containerName="authorino"
Jul 21 12:37:40.092310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.092254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c003f537-f826-44e9-91f6-148f78ffb946" containerName="authorino"
Jul 21 12:37:40.094109 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.094086    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.101807 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.101778    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7sbsq\""
Jul 21 12:37:40.101807 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.101796    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 21 12:37:40.102098 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.102081    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 21 12:37:40.102155 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.102109    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 21 12:37:40.112723 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.112703    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"]
Jul 21 12:37:40.188742 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.188714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.188869 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.188748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.188869 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.188765    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.188994 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.188871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.188994 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.188899    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdmjq\" (UniqueName: \"kubernetes.io/projected/426020a0-6085-4d58-8f72-506c0a9a2876-kube-api-access-mdmjq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.188994 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.188975    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426020a0-6085-4d58-8f72-506c0a9a2876-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.289708 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.289677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.289887 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.289746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.289887 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.289768    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdmjq\" (UniqueName: \"kubernetes.io/projected/426020a0-6085-4d58-8f72-506c0a9a2876-kube-api-access-mdmjq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.290045 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.289905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426020a0-6085-4d58-8f72-506c0a9a2876-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.290045 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.289986    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.290045 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.290027    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.290207 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.290182    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.290343 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.290322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.290393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.290378    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.292056 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.292037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/426020a0-6085-4d58-8f72-506c0a9a2876-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.292333 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.292316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/426020a0-6085-4d58-8f72-506c0a9a2876-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.298815 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.298791    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdmjq\" (UniqueName: \"kubernetes.io/projected/426020a0-6085-4d58-8f72-506c0a9a2876-kube-api-access-mdmjq\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm\" (UID: \"426020a0-6085-4d58-8f72-506c0a9a2876\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.404133 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.404049    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:40.542931 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:40.542908    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"]
Jul 21 12:37:40.545516 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:37:40.545476    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426020a0_6085_4d58_8f72_506c0a9a2876.slice/crio-d0a6a69dee1d12997aac6cadfb991a6145fd136aa91982aa5c19c60a08ec3c8d WatchSource:0}: Error finding container d0a6a69dee1d12997aac6cadfb991a6145fd136aa91982aa5c19c60a08ec3c8d: Status 404 returned error can't find the container with id d0a6a69dee1d12997aac6cadfb991a6145fd136aa91982aa5c19c60a08ec3c8d
Jul 21 12:37:41.296029 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:41.295993    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm" event={"ID":"426020a0-6085-4d58-8f72-506c0a9a2876","Type":"ContainerStarted","Data":"d0a6a69dee1d12997aac6cadfb991a6145fd136aa91982aa5c19c60a08ec3c8d"}
Jul 21 12:37:42.302295 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:42.302258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm" event={"ID":"426020a0-6085-4d58-8f72-506c0a9a2876","Type":"ContainerStarted","Data":"d1e05671bee666bd9fa83e2e653d73110a5119eb835cc75207b32939fbaa48cf"}
Jul 21 12:37:42.321754 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:42.321702    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm" podStartSLOduration=0.992369832 podStartE2EDuration="2.321686425s" totalImagesPullingTime="1.329316593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:37:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:37:42.320238723 +0000 UTC m=+600.105253714" watchObservedRunningTime="2026-07-21 12:37:42.321686425 +0000 UTC m=+600.106701390"
Jul 21 12:37:50.405229 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:50.405201    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:50.407117 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:50.405253    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:50.417817 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:50.417797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:51.356442 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.356412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm"
Jul 21 12:37:51.490878 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.490845    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"]
Jul 21 12:37:51.496799 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.496777    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.499460 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.499436    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 21 12:37:51.507767 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.507744    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"]
Jul 21 12:37:51.584266 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.584227    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.584528 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.584509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.584656 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.584639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnszh\" (UniqueName: \"kubernetes.io/projected/d227d846-f642-4883-92df-da804f6035dd-kube-api-access-qnszh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.584785 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.584770    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d227d846-f642-4883-92df-da804f6035dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.584950 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.584935    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.585107 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.585092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.685827 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.685741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d227d846-f642-4883-92df-da804f6035dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.685827 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.685802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.685827 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.685822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.686132 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.685854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.686132 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.686009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.686132 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.686045    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnszh\" (UniqueName: \"kubernetes.io/projected/d227d846-f642-4883-92df-da804f6035dd-kube-api-access-qnszh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.686293 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.686254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.686351 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.686287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.686463 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.686442    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.688221 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.688199    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d227d846-f642-4883-92df-da804f6035dd-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.688329 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.688311    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d227d846-f642-4883-92df-da804f6035dd-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.696438 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.696415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnszh\" (UniqueName: \"kubernetes.io/projected/d227d846-f642-4883-92df-da804f6035dd-kube-api-access-qnszh\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wblcn\" (UID: \"d227d846-f642-4883-92df-da804f6035dd\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.808661 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.808630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:37:51.946362 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.946339    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"]
Jul 21 12:37:51.948708 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:37:51.948682    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd227d846_f642_4883_92df_da804f6035dd.slice/crio-bd13c25897fc0b92817b7a79d710d0b058f41543219c1aff19266cc4216d3af3 WatchSource:0}: Error finding container bd13c25897fc0b92817b7a79d710d0b058f41543219c1aff19266cc4216d3af3: Status 404 returned error can't find the container with id bd13c25897fc0b92817b7a79d710d0b058f41543219c1aff19266cc4216d3af3
Jul 21 12:37:51.949919 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:51.949903    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 12:37:52.350463 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:52.350428    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn" event={"ID":"d227d846-f642-4883-92df-da804f6035dd","Type":"ContainerStarted","Data":"b2b93c32ae06c661949fda94ff73666a9438ee45117827bb79272c141244051c"}
Jul 21 12:37:52.350618 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:52.350479    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn" event={"ID":"d227d846-f642-4883-92df-da804f6035dd","Type":"ContainerStarted","Data":"bd13c25897fc0b92817b7a79d710d0b058f41543219c1aff19266cc4216d3af3"}
Jul 21 12:37:52.369119 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:37:52.369073    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn" podStartSLOduration=1.1824072860000001 podStartE2EDuration="1.369058695s" totalImagesPullingTime="186.651409ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:37:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:37:52.368802551 +0000 UTC m=+610.153817537" watchObservedRunningTime="2026-07-21 12:37:52.369058695 +0000 UTC m=+610.154073661"
Jul 21 12:38:01.809627 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:38:01.809580    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:38:01.810054 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:38:01.809686    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:38:01.821994 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:38:01.821952    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:38:02.406265 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:38:02.406240    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wblcn"
Jul 21 12:39:57.651366 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.651328    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb57694d8-rj9b2"]
Jul 21 12:39:57.652005 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.651524    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-cb57694d8-rj9b2" podUID="e570331f-3035-4c28-a599-769955d89d78" containerName="authorino" containerID="cri-o://730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb" gracePeriod=30
Jul 21 12:39:57.898866 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.898841    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:39:57.914170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.914080    2577 generic.go:356] "Generic (PLEG): container finished" podID="e570331f-3035-4c28-a599-769955d89d78" containerID="730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb" exitCode=0
Jul 21 12:39:57.914170 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.914148    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-cb57694d8-rj9b2"
Jul 21 12:39:57.914355 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.914152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb57694d8-rj9b2" event={"ID":"e570331f-3035-4c28-a599-769955d89d78","Type":"ContainerDied","Data":"730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb"}
Jul 21 12:39:57.914355 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.914304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-cb57694d8-rj9b2" event={"ID":"e570331f-3035-4c28-a599-769955d89d78","Type":"ContainerDied","Data":"2758751b0b0b427a27c9bbf07d088dfbf14c2d0c4f27908cffb27995ceba3c94"}
Jul 21 12:39:57.914355 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.914334    2577 scope.go:117] "RemoveContainer" containerID="730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb"
Jul 21 12:39:57.926729 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.926644    2577 scope.go:117] "RemoveContainer" containerID="730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb"
Jul 21 12:39:57.927829 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:39:57.927765    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb\": container with ID starting with 730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb not found: ID does not exist" containerID="730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb"
Jul 21 12:39:57.927829 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:57.927802    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb"} err="failed to get container status \"730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb\": rpc error: code = NotFound desc = could not find container \"730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb\": container with ID starting with 730f30ea0027eba89c814bc6fe4628a202a5a9dd71439b600953362e622a37fb not found: ID does not exist"
Jul 21 12:39:58.049404 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.049367    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96fkp\" (UniqueName: \"kubernetes.io/projected/e570331f-3035-4c28-a599-769955d89d78-kube-api-access-96fkp\") pod \"e570331f-3035-4c28-a599-769955d89d78\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") "
Jul 21 12:39:58.049559 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.049426    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e570331f-3035-4c28-a599-769955d89d78-tls-cert\") pod \"e570331f-3035-4c28-a599-769955d89d78\" (UID: \"e570331f-3035-4c28-a599-769955d89d78\") "
Jul 21 12:39:58.051460 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.051428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e570331f-3035-4c28-a599-769955d89d78-kube-api-access-96fkp" (OuterVolumeSpecName: "kube-api-access-96fkp") pod "e570331f-3035-4c28-a599-769955d89d78" (UID: "e570331f-3035-4c28-a599-769955d89d78"). InnerVolumeSpecName "kube-api-access-96fkp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:39:58.059421 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.059400    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e570331f-3035-4c28-a599-769955d89d78-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e570331f-3035-4c28-a599-769955d89d78" (UID: "e570331f-3035-4c28-a599-769955d89d78"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:39:58.150397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.150364    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e570331f-3035-4c28-a599-769955d89d78-tls-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:39:58.150397 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.150392    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96fkp\" (UniqueName: \"kubernetes.io/projected/e570331f-3035-4c28-a599-769955d89d78-kube-api-access-96fkp\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:39:58.241421 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.241393    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-cb57694d8-rj9b2"]
Jul 21 12:39:58.250358 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.250337    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-cb57694d8-rj9b2"]
Jul 21 12:39:58.832466 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:39:58.832436    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e570331f-3035-4c28-a599-769955d89d78" path="/var/lib/kubelet/pods/e570331f-3035-4c28-a599-769955d89d78/volumes"
Jul 21 12:46:52.990900 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:52.990861    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"]
Jul 21 12:46:52.991522 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:52.991145    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7" podUID="bb1203d6-dee2-4718-81fc-38f508fd5229" containerName="manager" containerID="cri-o://f667cc2a1cda40f111a5913ffac261b2cd996db3d19c74b3172154d2e6d5f902" gracePeriod=10
Jul 21 12:46:55.777224 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:55.777188    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb1203d6-dee2-4718-81fc-38f508fd5229" containerID="f667cc2a1cda40f111a5913ffac261b2cd996db3d19c74b3172154d2e6d5f902" exitCode=0
Jul 21 12:46:55.777587 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:55.777227    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7" event={"ID":"bb1203d6-dee2-4718-81fc-38f508fd5229","Type":"ContainerDied","Data":"f667cc2a1cda40f111a5913ffac261b2cd996db3d19c74b3172154d2e6d5f902"}
Jul 21 12:46:55.942610 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:55.942582    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:46:56.063068 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.062977    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjf7r\" (UniqueName: \"kubernetes.io/projected/bb1203d6-dee2-4718-81fc-38f508fd5229-kube-api-access-xjf7r\") pod \"bb1203d6-dee2-4718-81fc-38f508fd5229\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") "
Jul 21 12:46:56.063068 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.063063    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb1203d6-dee2-4718-81fc-38f508fd5229-extensions-socket-volume\") pod \"bb1203d6-dee2-4718-81fc-38f508fd5229\" (UID: \"bb1203d6-dee2-4718-81fc-38f508fd5229\") "
Jul 21 12:46:56.063435 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.063407    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb1203d6-dee2-4718-81fc-38f508fd5229-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bb1203d6-dee2-4718-81fc-38f508fd5229" (UID: "bb1203d6-dee2-4718-81fc-38f508fd5229"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:46:56.065010 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.064976    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb1203d6-dee2-4718-81fc-38f508fd5229-kube-api-access-xjf7r" (OuterVolumeSpecName: "kube-api-access-xjf7r") pod "bb1203d6-dee2-4718-81fc-38f508fd5229" (UID: "bb1203d6-dee2-4718-81fc-38f508fd5229"). InnerVolumeSpecName "kube-api-access-xjf7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:46:56.164147 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.164110    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjf7r\" (UniqueName: \"kubernetes.io/projected/bb1203d6-dee2-4718-81fc-38f508fd5229-kube-api-access-xjf7r\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:46:56.164147 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.164141    2577 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb1203d6-dee2-4718-81fc-38f508fd5229-extensions-socket-volume\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:46:56.782458 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.782426    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"
Jul 21 12:46:56.782849 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.782461    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7" event={"ID":"bb1203d6-dee2-4718-81fc-38f508fd5229","Type":"ContainerDied","Data":"008e12fbbb9bfcccff73f23c300d7a7250519764c8f7fb28c8697b15184e790e"}
Jul 21 12:46:56.782849 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.782519    2577 scope.go:117] "RemoveContainer" containerID="f667cc2a1cda40f111a5913ffac261b2cd996db3d19c74b3172154d2e6d5f902"
Jul 21 12:46:56.803353 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.803329    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"]
Jul 21 12:46:56.809225 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.809191    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-tmrt7"]
Jul 21 12:46:56.831715 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:46:56.831688    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb1203d6-dee2-4718-81fc-38f508fd5229" path="/var/lib/kubelet/pods/bb1203d6-dee2-4718-81fc-38f508fd5229/volumes"
Jul 21 12:47:54.167369 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.167334    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"]
Jul 21 12:47:54.167810 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.167676    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb1203d6-dee2-4718-81fc-38f508fd5229" containerName="manager"
Jul 21 12:47:54.167810 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.167687    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb1203d6-dee2-4718-81fc-38f508fd5229" containerName="manager"
Jul 21 12:47:54.167810 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.167698    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e570331f-3035-4c28-a599-769955d89d78" containerName="authorino"
Jul 21 12:47:54.167810 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.167704    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e570331f-3035-4c28-a599-769955d89d78" containerName="authorino"
Jul 21 12:47:54.167810 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.167767    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb1203d6-dee2-4718-81fc-38f508fd5229" containerName="manager"
Jul 21 12:47:54.167810 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.167776    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e570331f-3035-4c28-a599-769955d89d78" containerName="authorino"
Jul 21 12:47:54.171852 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.171835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.174544 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.174524    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-d4b6n\""
Jul 21 12:47:54.182614 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.182594    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"]
Jul 21 12:47:54.247498 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.247472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01f4cb0c-b4c6-48e2-bcd0-f899becae7b2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-65lgr\" (UID: \"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.247646 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.247516    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzp7j\" (UniqueName: \"kubernetes.io/projected/01f4cb0c-b4c6-48e2-bcd0-f899becae7b2-kube-api-access-hzp7j\") pod \"kuadrant-operator-controller-manager-86878c769f-65lgr\" (UID: \"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.348269 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.348237    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzp7j\" (UniqueName: \"kubernetes.io/projected/01f4cb0c-b4c6-48e2-bcd0-f899becae7b2-kube-api-access-hzp7j\") pod \"kuadrant-operator-controller-manager-86878c769f-65lgr\" (UID: \"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.348398 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.348307    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01f4cb0c-b4c6-48e2-bcd0-f899becae7b2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-65lgr\" (UID: \"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.348614 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.348599    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/01f4cb0c-b4c6-48e2-bcd0-f899becae7b2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-65lgr\" (UID: \"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.356628 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.356607    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzp7j\" (UniqueName: \"kubernetes.io/projected/01f4cb0c-b4c6-48e2-bcd0-f899becae7b2-kube-api-access-hzp7j\") pod \"kuadrant-operator-controller-manager-86878c769f-65lgr\" (UID: \"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.481523 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.481448    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:54.621875 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:54.621851    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"]
Jul 21 12:47:54.624545 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:47:54.624494    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01f4cb0c_b4c6_48e2_bcd0_f899becae7b2.slice/crio-7b0c0fd37b66771915cc8136c8cb6ea38249e057e8e86d0416ca53d3f94d1199 WatchSource:0}: Error finding container 7b0c0fd37b66771915cc8136c8cb6ea38249e057e8e86d0416ca53d3f94d1199: Status 404 returned error can't find the container with id 7b0c0fd37b66771915cc8136c8cb6ea38249e057e8e86d0416ca53d3f94d1199
Jul 21 12:47:55.033601 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:55.033568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr" event={"ID":"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2","Type":"ContainerStarted","Data":"0f9cb59fd334da1b9f62c52f9de7d8717632f86d020851b05cfba263fc1cac58"}
Jul 21 12:47:55.033770 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:55.033622    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr" event={"ID":"01f4cb0c-b4c6-48e2-bcd0-f899becae7b2","Type":"ContainerStarted","Data":"7b0c0fd37b66771915cc8136c8cb6ea38249e057e8e86d0416ca53d3f94d1199"}
Jul 21 12:47:55.033770 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:55.033657    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:47:55.075821 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:47:55.075768    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr" podStartSLOduration=1.075751186 podStartE2EDuration="1.075751186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:47:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:47:55.073184872 +0000 UTC m=+1212.858199838" watchObservedRunningTime="2026-07-21 12:47:55.075751186 +0000 UTC m=+1212.860766152"
Jul 21 12:48:06.040328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:48:06.040300    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-65lgr"
Jul 21 12:50:39.190945 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.190903    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"]
Jul 21 12:50:39.194202 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.194183    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.196827 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.196806    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-b6a8563f-kserve-self-signed-certs\""
Jul 21 12:50:39.217754 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.217734    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"]
Jul 21 12:50:39.269689 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.269652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cwqlk\" (UniqueName: \"kubernetes.io/projected/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-kube-api-access-cwqlk\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.269862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.269697    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-home\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.269862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.269716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tls-certs\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.269862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.269794    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-model-cache\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.269862 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.269850    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tmp-dir\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.270067 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.269901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-dshm\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.370704 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.370667    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tmp-dir\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.370880 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.370724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-dshm\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.370880 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.370780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cwqlk\" (UniqueName: \"kubernetes.io/projected/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-kube-api-access-cwqlk\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.370880 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.370817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-home\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.371094 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.370928    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tls-certs\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.371094 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.371015    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-model-cache\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.371198 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.371167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tmp-dir\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.371255 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.371242    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-home\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.371420 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.371399    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-model-cache\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.372967 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.372938    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-dshm\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.373526 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.373505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tls-certs\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.380337 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.380318    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cwqlk\" (UniqueName: \"kubernetes.io/projected/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-kube-api-access-cwqlk\") pod \"e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") " pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.503215 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.503191    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:39.634230 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.634203    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"]
Jul 21 12:50:39.636892 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:50:39.636856    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d5c5bd8_a9ca_4a4d_9848_52dd3204b3df.slice/crio-2d0ba23fd1add2b37dafab692c10e980df9a3ba0d411fd3c746c3dc16af0a876 WatchSource:0}: Error finding container 2d0ba23fd1add2b37dafab692c10e980df9a3ba0d411fd3c746c3dc16af0a876: Status 404 returned error can't find the container with id 2d0ba23fd1add2b37dafab692c10e980df9a3ba0d411fd3c746c3dc16af0a876
Jul 21 12:50:39.763835 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.763760    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr" event={"ID":"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df","Type":"ContainerStarted","Data":"a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561"}
Jul 21 12:50:39.763835 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.763799    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr" event={"ID":"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df","Type":"ContainerStarted","Data":"2d0ba23fd1add2b37dafab692c10e980df9a3ba0d411fd3c746c3dc16af0a876"}
Jul 21 12:50:39.782914 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:39.782871    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr" podStartSLOduration=0.782857495 podStartE2EDuration="782.857495ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:50:39.782131626 +0000 UTC m=+1377.567146593" watchObservedRunningTime="2026-07-21 12:50:39.782857495 +0000 UTC m=+1377.567872461"
Jul 21 12:50:49.504206 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:49.504171    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:49.504595 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:49.504221    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:49.516431 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:49.516407    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:49.821809 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:49.821728    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:50:52.984448 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:52.984409    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"]
Jul 21 12:50:52.988413 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:52.988388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:52.990552 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:52.990528    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-b6a8563f-kserve-self-signed-certs\""
Jul 21 12:50:52.997868 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:52.997847    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"]
Jul 21 12:50:53.089496 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.089464    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-model-cache\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.089496 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.089500    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7k4g\" (UniqueName: \"kubernetes.io/projected/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-kube-api-access-f7k4g\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.089682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.089541    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-dshm\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.089682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.089557    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tmp-dir\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.089682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.089657    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-home\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.089796 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.089724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tls-certs\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.190722 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.190677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tls-certs\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.190722 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.190730    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-model-cache\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.191006 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.190752    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7k4g\" (UniqueName: \"kubernetes.io/projected/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-kube-api-access-f7k4g\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.191006 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.190790    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-dshm\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.191006 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.190804    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tmp-dir\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.191006 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.190833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-home\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.191226 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.191195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-model-cache\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.191286 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.191269    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tmp-dir\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.191330 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.191288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-home\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.193111 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.193086    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-dshm\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.193265 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.193245    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tls-certs\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.198222 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.198198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7k4g\" (UniqueName: \"kubernetes.io/projected/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-kube-api-access-f7k4g\") pod \"e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") " pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.299021 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.298929    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:50:53.431864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.431832    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"]
Jul 21 12:50:53.435132 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:50:53.435101    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99f64837_7d55_4fb9_8d76_33e4f82e3ea1.slice/crio-e0be18754f82a2b959f7781c463f2c13ed6c2c2b3180deab71d1ff9b8370a7a8 WatchSource:0}: Error finding container e0be18754f82a2b959f7781c463f2c13ed6c2c2b3180deab71d1ff9b8370a7a8: Status 404 returned error can't find the container with id e0be18754f82a2b959f7781c463f2c13ed6c2c2b3180deab71d1ff9b8370a7a8
Jul 21 12:50:53.838456 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.838414    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd" event={"ID":"99f64837-7d55-4fb9-8d76-33e4f82e3ea1","Type":"ContainerStarted","Data":"8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d"}
Jul 21 12:50:53.838456 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.838455    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd" event={"ID":"99f64837-7d55-4fb9-8d76-33e4f82e3ea1","Type":"ContainerStarted","Data":"e0be18754f82a2b959f7781c463f2c13ed6c2c2b3180deab71d1ff9b8370a7a8"}
Jul 21 12:50:53.856175 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:50:53.856124    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd" podStartSLOduration=1.856109437 podStartE2EDuration="1.856109437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:50:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:50:53.853866874 +0000 UTC m=+1391.638881841" watchObservedRunningTime="2026-07-21 12:50:53.856109437 +0000 UTC m=+1391.641124402"
Jul 21 12:51:03.299520 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:03.299486    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:51:03.299892 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:03.299536    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:51:03.311577 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:03.311552    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:51:03.893118 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:03.893092    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:51:12.252708 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:12.252671    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"]
Jul 21 12:51:13.419708 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:13.419661    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"]
Jul 21 12:51:27.274581 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.274519    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr" podUID="1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" containerName="main" containerID="cri-o://a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561" gracePeriod=45
Jul 21 12:51:27.516398 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.516372    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:51:27.575800 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.575733    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-dshm\") pod \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") "
Jul 21 12:51:27.575800 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.575772    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tls-certs\") pod \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") "
Jul 21 12:51:27.575977 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.575813    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tmp-dir\") pod \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") "
Jul 21 12:51:27.575977 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.575840    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cwqlk\" (UniqueName: \"kubernetes.io/projected/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-kube-api-access-cwqlk\") pod \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") "
Jul 21 12:51:27.575977 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.575918    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-home\") pod \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") "
Jul 21 12:51:27.575977 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.575970    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-model-cache\") pod \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\" (UID: \"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df\") "
Jul 21 12:51:27.576187 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.576092    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" (UID: "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:27.576243 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.576198    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-home" (OuterVolumeSpecName: "home") pod "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" (UID: "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:27.576295 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.576232    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tmp-dir\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:27.576342 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.576318    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-model-cache" (OuterVolumeSpecName: "model-cache") pod "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" (UID: "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:27.577940 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.577919    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-dshm" (OuterVolumeSpecName: "dshm") pod "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" (UID: "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:27.578056 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.578013    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" (UID: "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:51:27.578056 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.578037    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-kube-api-access-cwqlk" (OuterVolumeSpecName: "kube-api-access-cwqlk") pod "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" (UID: "1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df"). InnerVolumeSpecName "kube-api-access-cwqlk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:51:27.677121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.677095    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-home\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:27.677121 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.677118    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-model-cache\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:27.677319 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.677128    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-dshm\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:27.677319 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.677136    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-tls-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:27.677319 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.677145    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cwqlk\" (UniqueName: \"kubernetes.io/projected/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df-kube-api-access-cwqlk\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:27.991154 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.991126    2577 generic.go:356] "Generic (PLEG): container finished" podID="1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" containerID="a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561" exitCode=0
Jul 21 12:51:27.991322 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.991173    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"
Jul 21 12:51:27.991322 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.991217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr" event={"ID":"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df","Type":"ContainerDied","Data":"a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561"}
Jul 21 12:51:27.991322 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.991258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr" event={"ID":"1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df","Type":"ContainerDied","Data":"2d0ba23fd1add2b37dafab692c10e980df9a3ba0d411fd3c746c3dc16af0a876"}
Jul 21 12:51:27.991322 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:27.991275    2577 scope.go:117] "RemoveContainer" containerID="a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561"
Jul 21 12:51:28.002747 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.002731    2577 scope.go:117] "RemoveContainer" containerID="a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561"
Jul 21 12:51:28.003027 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:51:28.003008    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561\": container with ID starting with a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561 not found: ID does not exist" containerID="a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561"
Jul 21 12:51:28.003119 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.003036    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561"} err="failed to get container status \"a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561\": rpc error: code = NotFound desc = could not find container \"a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561\": container with ID starting with a1b2305547f478e034e26735d74b9fdf0b9373f5c693993beac36411e3e7e561 not found: ID does not exist"
Jul 21 12:51:28.012942 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.012918    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"]
Jul 21 12:51:28.018695 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.018677    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-b6a8563f-kserve-667d5bf987-r54cr"]
Jul 21 12:51:28.441092 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.441003    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd" podUID="99f64837-7d55-4fb9-8d76-33e4f82e3ea1" containerName="main" containerID="cri-o://8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d" gracePeriod=45
Jul 21 12:51:28.689578 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.689555    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:51:28.785695 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.785670    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tls-certs\") pod \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") "
Jul 21 12:51:28.785872 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.785708    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7k4g\" (UniqueName: \"kubernetes.io/projected/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-kube-api-access-f7k4g\") pod \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") "
Jul 21 12:51:28.785872 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.785733    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tmp-dir\") pod \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") "
Jul 21 12:51:28.785872 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.785760    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-dshm\") pod \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") "
Jul 21 12:51:28.785872 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.785830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-model-cache\") pod \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") "
Jul 21 12:51:28.785872 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.785856    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-home\") pod \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\" (UID: \"99f64837-7d55-4fb9-8d76-33e4f82e3ea1\") "
Jul 21 12:51:28.786156 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.786021    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "99f64837-7d55-4fb9-8d76-33e4f82e3ea1" (UID: "99f64837-7d55-4fb9-8d76-33e4f82e3ea1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:28.786214 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.786155    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-model-cache" (OuterVolumeSpecName: "model-cache") pod "99f64837-7d55-4fb9-8d76-33e4f82e3ea1" (UID: "99f64837-7d55-4fb9-8d76-33e4f82e3ea1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:28.786214 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.786178    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-home" (OuterVolumeSpecName: "home") pod "99f64837-7d55-4fb9-8d76-33e4f82e3ea1" (UID: "99f64837-7d55-4fb9-8d76-33e4f82e3ea1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:28.786294 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.786214    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tmp-dir\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:28.787848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.787820    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-dshm" (OuterVolumeSpecName: "dshm") pod "99f64837-7d55-4fb9-8d76-33e4f82e3ea1" (UID: "99f64837-7d55-4fb9-8d76-33e4f82e3ea1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:51:28.787987 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.787864    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "99f64837-7d55-4fb9-8d76-33e4f82e3ea1" (UID: "99f64837-7d55-4fb9-8d76-33e4f82e3ea1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 12:51:28.788055 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.787986    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-kube-api-access-f7k4g" (OuterVolumeSpecName: "kube-api-access-f7k4g") pod "99f64837-7d55-4fb9-8d76-33e4f82e3ea1" (UID: "99f64837-7d55-4fb9-8d76-33e4f82e3ea1"). InnerVolumeSpecName "kube-api-access-f7k4g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:51:28.832776 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.832747    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" path="/var/lib/kubelet/pods/1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df/volumes"
Jul 21 12:51:28.887553 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.887529    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-tls-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:28.887553 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.887550    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7k4g\" (UniqueName: \"kubernetes.io/projected/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-kube-api-access-f7k4g\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:28.887686 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.887561    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-dshm\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:28.887686 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.887570    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-model-cache\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:28.887686 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.887578    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/99f64837-7d55-4fb9-8d76-33e4f82e3ea1-home\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:51:28.996741 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.996711    2577 generic.go:356] "Generic (PLEG): container finished" podID="99f64837-7d55-4fb9-8d76-33e4f82e3ea1" containerID="8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d" exitCode=0
Jul 21 12:51:28.996885 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.996752    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"
Jul 21 12:51:28.996885 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.996805    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd" event={"ID":"99f64837-7d55-4fb9-8d76-33e4f82e3ea1","Type":"ContainerDied","Data":"8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d"}
Jul 21 12:51:28.996885 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.996845    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd" event={"ID":"99f64837-7d55-4fb9-8d76-33e4f82e3ea1","Type":"ContainerDied","Data":"e0be18754f82a2b959f7781c463f2c13ed6c2c2b3180deab71d1ff9b8370a7a8"}
Jul 21 12:51:28.996885 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:28.996861    2577 scope.go:117] "RemoveContainer" containerID="8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d"
Jul 21 12:51:29.005057 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:29.005040    2577 scope.go:117] "RemoveContainer" containerID="8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d"
Jul 21 12:51:29.005328 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:51:29.005302    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d\": container with ID starting with 8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d not found: ID does not exist" containerID="8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d"
Jul 21 12:51:29.005394 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:29.005336    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d"} err="failed to get container status \"8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d\": rpc error: code = NotFound desc = could not find container \"8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d\": container with ID starting with 8cc57f9ba427c2a152c32a9a590fb57e220eb080b72204352691b1fd7fdfe16d not found: ID does not exist"
Jul 21 12:51:29.014083 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:29.014056    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"]
Jul 21 12:51:29.016071 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:29.016052    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-b6a8563f-kserve-59ff7569cf-vwvqd"]
Jul 21 12:51:30.837625 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:51:30.837591    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99f64837-7d55-4fb9-8d76-33e4f82e3ea1" path="/var/lib/kubelet/pods/99f64837-7d55-4fb9-8d76-33e4f82e3ea1/volumes"
Jul 21 12:54:31.514582 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.514543    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"]
Jul 21 12:54:31.515069 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.515040    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99f64837-7d55-4fb9-8d76-33e4f82e3ea1" containerName="main"
Jul 21 12:54:31.515069 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.515057    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="99f64837-7d55-4fb9-8d76-33e4f82e3ea1" containerName="main"
Jul 21 12:54:31.515145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.515082    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" containerName="main"
Jul 21 12:54:31.515145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.515092    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" containerName="main"
Jul 21 12:54:31.515216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.515172    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="99f64837-7d55-4fb9-8d76-33e4f82e3ea1" containerName="main"
Jul 21 12:54:31.515216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.515188    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d5c5bd8-a9ca-4a4d-9848-52dd3204b3df" containerName="main"
Jul 21 12:54:31.518685 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.518659    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.521826 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.521804    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-9e430475-openshift-default-dockercfg-nsndh\""
Jul 21 12:54:31.596736 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.596687    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istiod-ca-cert\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.596736 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.596740    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-podinfo\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.597018 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.596838    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-token\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.597018 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.596889    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-socket\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.597018 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.596916    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7xhc\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-kube-api-access-m7xhc\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.597018 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.596950    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-envoy\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.597186 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.597035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-data\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.597186 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.597064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-certs\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.597186 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.597087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-credential-socket\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.608688 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.608654    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"]
Jul 21 12:54:31.698026 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.697990    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istiod-ca-cert\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698026 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-podinfo\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698268 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-token\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698268 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-socket\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698268 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698223    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7xhc\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-kube-api-access-m7xhc\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698268 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698265    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-envoy\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698487 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-data\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698487 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698338    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-certs\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698487 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698375    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-credential-socket\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698699 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698673    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-socket\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698763 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-data\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698763 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698725    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-credential-socket\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.698868 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.698789    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-certs\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.699067 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.699043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istiod-ca-cert\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.700535 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.700515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-envoy\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.700736 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.700718    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-podinfo\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.708694 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.708659    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7xhc\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-kube-api-access-m7xhc\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.709820 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.709802    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-token\") pod \"e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") " pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.833889 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.833815    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:31.972192 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.972161    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"]
Jul 21 12:54:31.975573 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:54:31.975541    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea8ac76a_1c1e_498e_ad51_5bd850d83a2e.slice/crio-21a04c0a43ccef9f792a7926be8a141ff6d77d3e1666573fde3d1317b44b09df WatchSource:0}: Error finding container 21a04c0a43ccef9f792a7926be8a141ff6d77d3e1666573fde3d1317b44b09df: Status 404 returned error can't find the container with id 21a04c0a43ccef9f792a7926be8a141ff6d77d3e1666573fde3d1317b44b09df
Jul 21 12:54:31.977563 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.977528    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:54:31.977647 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.977593    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:54:31.977647 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:31.977621    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:54:32.810238 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:32.810205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" event={"ID":"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e","Type":"ContainerStarted","Data":"292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4"}
Jul 21 12:54:32.810592 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:32.810253    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" event={"ID":"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e","Type":"ContainerStarted","Data":"21a04c0a43ccef9f792a7926be8a141ff6d77d3e1666573fde3d1317b44b09df"}
Jul 21 12:54:32.834331 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:32.834284    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" podStartSLOduration=1.834269064 podStartE2EDuration="1.834269064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:54:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:54:32.833240446 +0000 UTC m=+1610.618255422" watchObservedRunningTime="2026-07-21 12:54:32.834269064 +0000 UTC m=+1610.619284029"
Jul 21 12:54:32.835844 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:32.835824    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:33.839125 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:33.839095    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:34.819287 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:34.819258    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:34.820207 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:34.820188    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:48.884498 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:48.884466    2577 generic.go:356] "Generic (PLEG): container finished" podID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerID="292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4" exitCode=0
Jul 21 12:54:48.884868 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:48.884538    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" event={"ID":"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e","Type":"ContainerDied","Data":"292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4"}
Jul 21 12:54:48.884981 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:48.884944    2577 scope.go:117] "RemoveContainer" containerID="292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4"
Jul 21 12:54:48.885940 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:48.885912    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:54:48.886059 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:48.886004    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:54:48.886059 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:48.886040    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:54:49.819351 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:49.819316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:49.891187 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:49.891155    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" event={"ID":"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e","Type":"ContainerStarted","Data":"56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add"}
Jul 21 12:54:50.835337 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:50.835304    2577 patch_prober.go:28] interesting pod/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:54:50.835617 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:50.835551    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:50.835755 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:50.835605    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:54:51.834975 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:51.834917    2577 patch_prober.go:28] interesting pod/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:54:51.835469 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:51.835001    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:54:52.856440 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:52.856403    2577 patch_prober.go:28] interesting pod/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:54:52.856912 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:52.856478    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:54:53.835346 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:53.835299    2577 patch_prober.go:28] interesting pod/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body=
Jul 21 12:54:53.835598 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:53.835379    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused"
Jul 21 12:54:54.838503 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:54.838474    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:54.838911 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:54.838736    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:54:54.839381 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:54:54.839363    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:55:15.136879 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:15.136841    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"]
Jul 21 12:55:15.137441 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:15.137077    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy" containerID="cri-o://56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add" gracePeriod=30
Jul 21 12:55:16.195033 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.194991    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"]
Jul 21 12:55:16.198237 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.198218    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.200730 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.200708    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-cf78342b-gw-openshift-default-dockercfg-jzczd\""
Jul 21 12:55:16.214355 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.214330    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"]
Jul 21 12:55:16.288938 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.288893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-certs\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289129 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.288982    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzq22\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-kube-api-access-tzq22\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289129 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.289046    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istiod-ca-cert\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289129 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.289082    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-podinfo\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289129 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.289104    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-token\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289289 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.289199    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-credential-socket\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289289 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.289228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-data\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289289 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.289262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-socket\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.289442 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.289295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-envoy\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390026 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.389992    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-credential-socket\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390031    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-data\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390063    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-socket\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-envoy\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390123    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-certs\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390161    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzq22\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-kube-api-access-tzq22\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istiod-ca-cert\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390512 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-podinfo\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390512 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390295    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-token\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390512 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390485    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-credential-socket\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390538    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-data\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390682 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390557    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-socket\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.390798 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.390776    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-certs\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.391216 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.391191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istiod-ca-cert\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.392554 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.392530    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-envoy\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.393016 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.393000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-podinfo\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.399889 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.399857    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-token\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.399889 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.399878    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzq22\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-kube-api-access-tzq22\") pod \"e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") " pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.509008 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.508927    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:16.658848 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.658819    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"]
Jul 21 12:55:16.661592 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:55:16.661562    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb83e198_abb2_41ee_b8f9_41fd5cf8139f.slice/crio-3ff98b7d8fe80f21ee30e391777c5fe081b220e8493abd9d5068a617bbce5dc4 WatchSource:0}: Error finding container 3ff98b7d8fe80f21ee30e391777c5fe081b220e8493abd9d5068a617bbce5dc4: Status 404 returned error can't find the container with id 3ff98b7d8fe80f21ee30e391777c5fe081b220e8493abd9d5068a617bbce5dc4
Jul 21 12:55:16.664372 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.664338    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:55:16.664469 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.664423    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:55:16.664469 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:16.664461    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:55:17.021305 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:17.021272    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" event={"ID":"bb83e198-abb2-41ee-b8f9-41fd5cf8139f","Type":"ContainerStarted","Data":"3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa"}
Jul 21 12:55:17.021466 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:17.021324    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" event={"ID":"bb83e198-abb2-41ee-b8f9-41fd5cf8139f","Type":"ContainerStarted","Data":"3ff98b7d8fe80f21ee30e391777c5fe081b220e8493abd9d5068a617bbce5dc4"}
Jul 21 12:55:17.092221 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:17.092169    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" podStartSLOduration=1.092152248 podStartE2EDuration="1.092152248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:55:17.089131577 +0000 UTC m=+1654.874146543" watchObservedRunningTime="2026-07-21 12:55:17.092152248 +0000 UTC m=+1654.877167213"
Jul 21 12:55:17.509698 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:17.509661    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:17.514458 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:17.514432    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:18.026883 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:18.026851    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:18.027847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:18.027825    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:20.385452 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.385425    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:55:20.431617 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431520    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-credential-socket\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431617 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431570    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-envoy\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431617 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431605    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-data\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431685    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m7xhc\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-kube-api-access-m7xhc\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431727    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istiod-ca-cert\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-socket\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431786    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-certs\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431814    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-token\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.431904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431832    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:20.431904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.431873    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-podinfo\") pod \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\" (UID: \"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e\") "
Jul 21 12:55:20.432296 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.432012    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:20.432296 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.432070    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:20.432296 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.432156    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.432296 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.432177    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.432296 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.432152    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:55:20.432782 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.432755    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:20.434361 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.434333    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:55:20.434644 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.434611    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-kube-api-access-m7xhc" (OuterVolumeSpecName: "kube-api-access-m7xhc") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "kube-api-access-m7xhc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:55:20.434644 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.434617    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:55:20.434777 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.434693    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" (UID: "ea8ac76a-1c1e-498e-ad51-5bd850d83a2e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:55:20.533483 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.533447    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.533483 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.533475    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.533483 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.533484    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.533483 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.533492    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.533823 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.533500    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.533823 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.533509    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:20.533823 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:20.533519    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m7xhc\" (UniqueName: \"kubernetes.io/projected/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e-kube-api-access-m7xhc\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:55:21.041716 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.041682    2577 generic.go:356] "Generic (PLEG): container finished" podID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerID="56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add" exitCode=0
Jul 21 12:55:21.041904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.041746    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"
Jul 21 12:55:21.041904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.041772    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" event={"ID":"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e","Type":"ContainerDied","Data":"56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add"}
Jul 21 12:55:21.041904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.041869    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t" event={"ID":"ea8ac76a-1c1e-498e-ad51-5bd850d83a2e","Type":"ContainerDied","Data":"21a04c0a43ccef9f792a7926be8a141ff6d77d3e1666573fde3d1317b44b09df"}
Jul 21 12:55:21.041904 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.041899    2577 scope.go:117] "RemoveContainer" containerID="56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add"
Jul 21 12:55:21.050393 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.050376    2577 scope.go:117] "RemoveContainer" containerID="292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4"
Jul 21 12:55:21.057939 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.057911    2577 scope.go:117] "RemoveContainer" containerID="56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add"
Jul 21 12:55:21.058311 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:55:21.058283    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add\": container with ID starting with 56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add not found: ID does not exist" containerID="56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add"
Jul 21 12:55:21.058417 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.058321    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add"} err="failed to get container status \"56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add\": rpc error: code = NotFound desc = could not find container \"56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add\": container with ID starting with 56240c35ac2dfb5e3dedadddc90fc3ecc2abe688abc26b0e4c5c8d52e6512add not found: ID does not exist"
Jul 21 12:55:21.058417 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.058344    2577 scope.go:117] "RemoveContainer" containerID="292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4"
Jul 21 12:55:21.058656 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:55:21.058636    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4\": container with ID starting with 292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4 not found: ID does not exist" containerID="292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4"
Jul 21 12:55:21.058743 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.058662    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4"} err="failed to get container status \"292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4\": rpc error: code = NotFound desc = could not find container \"292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4\": container with ID starting with 292c5aef839574bf791207af8f3a5866a3f1b0a78f275fa27bf5836993181bf4 not found: ID does not exist"
Jul 21 12:55:21.060385 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.060367    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"]
Jul 21 12:55:21.068561 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:21.068541    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-9e430475-openshift-default-7f97cdf64f-2z88t"]
Jul 21 12:55:22.832788 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:22.832755    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" path="/var/lib/kubelet/pods/ea8ac76a-1c1e-498e-ad51-5bd850d83a2e/volumes"
Jul 21 12:55:32.093512 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:32.093479    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerID="3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa" exitCode=0
Jul 21 12:55:32.093926 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:32.093553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" event={"ID":"bb83e198-abb2-41ee-b8f9-41fd5cf8139f","Type":"ContainerDied","Data":"3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa"}
Jul 21 12:55:32.093926 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:32.093919    2577 scope.go:117] "RemoveContainer" containerID="3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa"
Jul 21 12:55:32.094977 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:32.094927    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:55:32.095053 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:32.095019    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:55:32.095104 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:32.095053    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:55:33.027544 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:33.027497    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:33.099829 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:33.099794    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" event={"ID":"bb83e198-abb2-41ee-b8f9-41fd5cf8139f","Type":"ContainerStarted","Data":"14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65"}
Jul 21 12:55:33.509320 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:33.509281    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:33.510489 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:33.510461    2577 patch_prober.go:28] interesting pod/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:55:33.510608 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:33.510511    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 21 12:55:34.509895 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:34.509854    2577 patch_prober.go:28] interesting pod/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:55:34.510330 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:34.509910    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 21 12:55:35.509535 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:35.509497    2577 patch_prober.go:28] interesting pod/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:55:35.509710 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:35.509556    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 21 12:55:36.509798 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:36.509760    2577 patch_prober.go:28] interesting pod/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body=
Jul 21 12:55:36.510242 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:36.509817    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused"
Jul 21 12:55:37.513875 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:37.513845    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:37.514272 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:37.514206    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:37.515002 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:37.514983    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:55:58.918172 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:58.918140    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"]
Jul 21 12:55:58.918672 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:58.918343    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy" containerID="cri-o://14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65" gracePeriod=30
Jul 21 12:55:59.753688 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.753655    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"]
Jul 21 12:55:59.754006 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.753994    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy"
Jul 21 12:55:59.754058 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.754008    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy"
Jul 21 12:55:59.754058 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.754031    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy"
Jul 21 12:55:59.754058 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.754036    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy"
Jul 21 12:55:59.754147 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.754089    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy"
Jul 21 12:55:59.754147 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.754099    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea8ac76a-1c1e-498e-ad51-5bd850d83a2e" containerName="istio-proxy"
Jul 21 12:55:59.756290 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.756274    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.759729 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.759709    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-0645c77a-openshift-default-dockercfg-jlcpn\""
Jul 21 12:55:59.765037 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765017    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-socket\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765128 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765049    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-credential-socket\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765128 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-data\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765224 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765158    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-certs\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765224 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-token\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765292 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765263    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-podinfo\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765329 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765306    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istiod-ca-cert\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765369 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxfsg\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-kube-api-access-bxfsg\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.765410 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.765373    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-envoy\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.793276 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.793254    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"]
Jul 21 12:55:59.865774 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bxfsg\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-kube-api-access-bxfsg\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.865774 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-envoy\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866047 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865824    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-socket\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866047 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865840    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-credential-socket\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866047 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-data\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866047 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-certs\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866047 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865930    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-token\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866047 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.865980    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-podinfo\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866047 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.866011    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istiod-ca-cert\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866396 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.866345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-data\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866484 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.866426    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-credential-socket\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866564 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.866547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-socket\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866623 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.866547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-certs\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.866710 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.866688    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istiod-ca-cert\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.868300 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.868277    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-envoy\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.868617 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.868596    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-podinfo\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.891634 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.891609    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxfsg\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-kube-api-access-bxfsg\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:55:59.894363 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:55:59.894341    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-token\") pod \"e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") " pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:00.066531 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:00.066452    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:00.213109 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:00.213080    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"]
Jul 21 12:56:00.216077 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:56:00.216050    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb114d5a1_53a1_4ca1_a7b1_12990b1e5251.slice/crio-a6cbd647c7e5935f7ef42d70b6a5707f8aad9a2c64f108735c5dd9d2fad4542f WatchSource:0}: Error finding container a6cbd647c7e5935f7ef42d70b6a5707f8aad9a2c64f108735c5dd9d2fad4542f: Status 404 returned error can't find the container with id a6cbd647c7e5935f7ef42d70b6a5707f8aad9a2c64f108735c5dd9d2fad4542f
Jul 21 12:56:00.218079 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:00.218047    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:00.218163 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:00.218110    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:00.218163 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:00.218140    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:00.223776 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:00.223749    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" event={"ID":"b114d5a1-53a1-4ca1-a7b1-12990b1e5251","Type":"ContainerStarted","Data":"a6cbd647c7e5935f7ef42d70b6a5707f8aad9a2c64f108735c5dd9d2fad4542f"}
Jul 21 12:56:01.231388 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:01.231341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" event={"ID":"b114d5a1-53a1-4ca1-a7b1-12990b1e5251","Type":"ContainerStarted","Data":"bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161"}
Jul 21 12:56:02.067042 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:02.067013    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:02.071259 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:02.071237    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:02.121489 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:02.121435    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" podStartSLOduration=3.121420289 podStartE2EDuration="3.121420289s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:55:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:56:01.272833948 +0000 UTC m=+1699.057848914" watchObservedRunningTime="2026-07-21 12:56:02.121420289 +0000 UTC m=+1699.906435254"
Jul 21 12:56:02.239012 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:02.238978    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:02.239807 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:02.239787    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:04.175758 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.175729    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:56:04.201459 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201367    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-data\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.201459 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201412    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-certs\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.201459 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201432    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-podinfo\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.201744 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201467    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzq22\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-kube-api-access-tzq22\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.201744 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201500    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istiod-ca-cert\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.201744 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201521    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-credential-socket\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.201744 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201555    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-socket\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.201744 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201722    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:04.202009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201834    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:04.202009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201852    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:04.202009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201870    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-token\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.202009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201906    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-envoy\") pod \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\" (UID: \"bb83e198-abb2-41ee-b8f9-41fd5cf8139f\") "
Jul 21 12:56:04.202009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.201910    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:56:04.202310 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.202291    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.202366 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.202315    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.202366 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.202328    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.202366 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.202350    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.202552 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.202500    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:04.203902 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.203881    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-kube-api-access-tzq22" (OuterVolumeSpecName: "kube-api-access-tzq22") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "kube-api-access-tzq22". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:56:04.204007 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.203915    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:04.204181 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.204151    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:56:04.204181 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.204163    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb83e198-abb2-41ee-b8f9-41fd5cf8139f" (UID: "bb83e198-abb2-41ee-b8f9-41fd5cf8139f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:56:04.250559 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.250521    2577 generic.go:356] "Generic (PLEG): container finished" podID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerID="14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65" exitCode=0
Jul 21 12:56:04.250730 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.250571    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"
Jul 21 12:56:04.250730 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.250604    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" event={"ID":"bb83e198-abb2-41ee-b8f9-41fd5cf8139f","Type":"ContainerDied","Data":"14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65"}
Jul 21 12:56:04.250730 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.250661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp" event={"ID":"bb83e198-abb2-41ee-b8f9-41fd5cf8139f","Type":"ContainerDied","Data":"3ff98b7d8fe80f21ee30e391777c5fe081b220e8493abd9d5068a617bbce5dc4"}
Jul 21 12:56:04.250730 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.250692    2577 scope.go:117] "RemoveContainer" containerID="14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65"
Jul 21 12:56:04.259804 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.259786    2577 scope.go:117] "RemoveContainer" containerID="3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa"
Jul 21 12:56:04.267577 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.267558    2577 scope.go:117] "RemoveContainer" containerID="14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65"
Jul 21 12:56:04.267866 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:56:04.267849    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65\": container with ID starting with 14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65 not found: ID does not exist" containerID="14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65"
Jul 21 12:56:04.267933 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.267874    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65"} err="failed to get container status \"14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65\": rpc error: code = NotFound desc = could not find container \"14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65\": container with ID starting with 14364a49a8ab7f76de3b878e586f51823d9d6d23a94afc80f7fa654439b14e65 not found: ID does not exist"
Jul 21 12:56:04.267933 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.267892    2577 scope.go:117] "RemoveContainer" containerID="3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa"
Jul 21 12:56:04.268165 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:56:04.268142    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa\": container with ID starting with 3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa not found: ID does not exist" containerID="3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa"
Jul 21 12:56:04.268261 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.268171    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa"} err="failed to get container status \"3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa\": rpc error: code = NotFound desc = could not find container \"3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa\": container with ID starting with 3c8a2b438a5741d57b22729aabdd396270ee8a66c9ec073bff61087019d2b5fa not found: ID does not exist"
Jul 21 12:56:04.292709 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.292672    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"]
Jul 21 12:56:04.299715 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.299687    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-cf78342b-gw-openshift-default-5bf755c68c-lbwsp"]
Jul 21 12:56:04.303808 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.303787    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.303808 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.303810    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.303971 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.303824    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzq22\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-kube-api-access-tzq22\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.303971 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.303837    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.303971 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.303848    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb83e198-abb2-41ee-b8f9-41fd5cf8139f-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:04.835124 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:04.835080    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" path="/var/lib/kubelet/pods/bb83e198-abb2-41ee-b8f9-41fd5cf8139f/volumes"
Jul 21 12:56:16.308002 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:16.307946    2577 generic.go:356] "Generic (PLEG): container finished" podID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerID="bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161" exitCode=0
Jul 21 12:56:16.308442 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:16.308016    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" event={"ID":"b114d5a1-53a1-4ca1-a7b1-12990b1e5251","Type":"ContainerDied","Data":"bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161"}
Jul 21 12:56:16.308442 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:16.308415    2577 scope.go:117] "RemoveContainer" containerID="bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161"
Jul 21 12:56:16.309370 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:16.309342    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:16.309429 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:16.309416    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:16.309470 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:16.309449    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:17.239095 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:17.239046    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:17.314340 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:17.314304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" event={"ID":"b114d5a1-53a1-4ca1-a7b1-12990b1e5251","Type":"ContainerStarted","Data":"ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81"}
Jul 21 12:56:18.067305 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:18.067259    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:18.068448 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:18.068418    2577 patch_prober.go:28] interesting pod/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:56:18.068593 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:18.068478    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:56:19.067358 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:19.067320    2577 patch_prober.go:28] interesting pod/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:56:19.067780 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:19.067378    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:56:20.067380 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:20.067342    2577 patch_prober.go:28] interesting pod/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:56:20.067793 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:20.067402    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:56:21.066979 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:21.066918    2577 patch_prober.go:28] interesting pod/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body=
Jul 21 12:56:21.067188 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:21.067000    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused"
Jul 21 12:56:22.071401 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:22.071370    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:22.071800 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:22.071649    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:22.072410 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:22.072391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:44.918680 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:44.918640    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"]
Jul 21 12:56:44.919361 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:44.918998    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy" containerID="cri-o://ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81" gracePeriod=30
Jul 21 12:56:45.733396 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.733361    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"]
Jul 21 12:56:45.733705 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.733692    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy"
Jul 21 12:56:45.733705 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.733705    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy"
Jul 21 12:56:45.733825 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.733770    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy"
Jul 21 12:56:45.733825 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.733781    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy"
Jul 21 12:56:45.736776 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.736756    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.742767 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.742746    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-6b5db8f8-openshift-default-dockercfg-cc5p5\""
Jul 21 12:56:45.754785 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.754759    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"]
Jul 21 12:56:45.877908 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.877873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-socket\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.877908 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.877909    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-credential-socket\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.878188 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.877932    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-envoy\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.878188 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.878009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-data\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.878188 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.878100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istiod-ca-cert\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.878188 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.878154    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6w65\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-kube-api-access-v6w65\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.878332 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.878200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-token\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.878332 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.878249    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-podinfo\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.878395 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.878365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-certs\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.979576 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.979541    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-token\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.979589    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-podinfo\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.979669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-certs\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.979799    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-socket\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.979851    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-credential-socket\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.979876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-envoy\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.979949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-data\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.980013    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istiod-ca-cert\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980046 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.980038    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6w65\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-kube-api-access-v6w65\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980432 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.980097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-certs\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980432 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.980250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-socket\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980432 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.980251    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-credential-socket\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980432 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.980330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-data\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.980778 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.980753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istiod-ca-cert\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.982082 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.982062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-envoy\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.982314 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.982295    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-podinfo\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.990713 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.990657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-token\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:45.990819 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:45.990797    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6w65\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-kube-api-access-v6w65\") pod \"e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") " pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:46.048740 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.048712    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:46.200259 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.200231    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"]
Jul 21 12:56:46.201886 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:56:46.201865    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddafb9848_08ee_4dac_8004_fe8b2cd747b2.slice/crio-54aa4994cc6f9801d5fae328ddbffc2be2ca4d18da54edabeffed8d754b9717d WatchSource:0}: Error finding container 54aa4994cc6f9801d5fae328ddbffc2be2ca4d18da54edabeffed8d754b9717d: Status 404 returned error can't find the container with id 54aa4994cc6f9801d5fae328ddbffc2be2ca4d18da54edabeffed8d754b9717d
Jul 21 12:56:46.204134 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.204099    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:46.204245 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.204180    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:46.204245 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.204224    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:56:46.454945 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.454912    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" event={"ID":"dafb9848-08ee-4dac-8004-fe8b2cd747b2","Type":"ContainerStarted","Data":"e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7"}
Jul 21 12:56:46.455129 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.454988    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" event={"ID":"dafb9848-08ee-4dac-8004-fe8b2cd747b2","Type":"ContainerStarted","Data":"54aa4994cc6f9801d5fae328ddbffc2be2ca4d18da54edabeffed8d754b9717d"}
Jul 21 12:56:46.482354 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:46.482303    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" podStartSLOduration=1.48228827 podStartE2EDuration="1.48228827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:56:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:56:46.480400617 +0000 UTC m=+1744.265415584" watchObservedRunningTime="2026-07-21 12:56:46.48228827 +0000 UTC m=+1744.267303240"
Jul 21 12:56:47.049600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:47.049572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:47.053888 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:47.053863    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:47.459264 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:47.459184    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:47.460065 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:47.460047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:56:50.193141 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.193082    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:50.212555 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-podinfo\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.212689 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212581    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-envoy\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.212689 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-data\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.212689 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212664    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-certs\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.212847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212688    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bxfsg\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-kube-api-access-bxfsg\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.212847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212718    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-socket\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.212847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212744    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-credential-socket\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.212847 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212780    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-token\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.213078 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.212908    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:50.213136 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.213073    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.213191 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.213161    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:50.213253 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.213222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:50.213697 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.213670    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-data" (OuterVolumeSpecName: "istio-data") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:50.215221 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.215186    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-kube-api-access-bxfsg" (OuterVolumeSpecName: "kube-api-access-bxfsg") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "kube-api-access-bxfsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:56:50.215774 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.215719    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:56:50.215987 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.215931    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-token" (OuterVolumeSpecName: "istio-token") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:56:50.215987 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.215939    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:56:50.314026 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.313997    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istiod-ca-cert\") pod \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\" (UID: \"b114d5a1-53a1-4ca1-a7b1-12990b1e5251\") "
Jul 21 12:56:50.314189 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314161    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.314189 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314172    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.314189 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314181    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.314189 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314190    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.314324 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314198    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.314324 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314207    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.314324 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314214    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bxfsg\" (UniqueName: \"kubernetes.io/projected/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-kube-api-access-bxfsg\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.314412 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.314329    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b114d5a1-53a1-4ca1-a7b1-12990b1e5251" (UID: "b114d5a1-53a1-4ca1-a7b1-12990b1e5251"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:56:50.415601 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.415576    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b114d5a1-53a1-4ca1-a7b1-12990b1e5251-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:56:50.473709 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.473628    2577 generic.go:356] "Generic (PLEG): container finished" podID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerID="ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81" exitCode=0
Jul 21 12:56:50.473709 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.473683    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"
Jul 21 12:56:50.473709 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.473692    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" event={"ID":"b114d5a1-53a1-4ca1-a7b1-12990b1e5251","Type":"ContainerDied","Data":"ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81"}
Jul 21 12:56:50.473899 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.473764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5" event={"ID":"b114d5a1-53a1-4ca1-a7b1-12990b1e5251","Type":"ContainerDied","Data":"a6cbd647c7e5935f7ef42d70b6a5707f8aad9a2c64f108735c5dd9d2fad4542f"}
Jul 21 12:56:50.473899 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.473788    2577 scope.go:117] "RemoveContainer" containerID="ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81"
Jul 21 12:56:50.482863 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.482846    2577 scope.go:117] "RemoveContainer" containerID="bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161"
Jul 21 12:56:50.489996 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.489982    2577 scope.go:117] "RemoveContainer" containerID="ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81"
Jul 21 12:56:50.490220 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:56:50.490204    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81\": container with ID starting with ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81 not found: ID does not exist" containerID="ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81"
Jul 21 12:56:50.490279 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.490226    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81"} err="failed to get container status \"ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81\": rpc error: code = NotFound desc = could not find container \"ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81\": container with ID starting with ea1ef4288966c2e2aee1f029ef6cd10ee6db23cf8d104cfd7b22d169d1847b81 not found: ID does not exist"
Jul 21 12:56:50.490279 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.490241    2577 scope.go:117] "RemoveContainer" containerID="bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161"
Jul 21 12:56:50.490428 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:56:50.490413    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161\": container with ID starting with bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161 not found: ID does not exist" containerID="bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161"
Jul 21 12:56:50.490470 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.490431    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161"} err="failed to get container status \"bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161\": rpc error: code = NotFound desc = could not find container \"bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161\": container with ID starting with bb6d9ffdabb1f5fca3effda92ddecf188d6412038755d81aa1c9058c8fd24161 not found: ID does not exist"
Jul 21 12:56:50.495937 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.495916    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"]
Jul 21 12:56:50.502396 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.502373    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-0645c77a-openshift-default-84897488bf-jk7z5"]
Jul 21 12:56:50.832464 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:56:50.832431    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" path="/var/lib/kubelet/pods/b114d5a1-53a1-4ca1-a7b1-12990b1e5251/volumes"
Jul 21 12:57:02.460527 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.460492    2577 patch_prober.go:28] interesting pod/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 21 12:57:02.460937 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.460560    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 21 12:57:02.536843 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.536810    2577 generic.go:356] "Generic (PLEG): container finished" podID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerID="e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7" exitCode=0
Jul 21 12:57:02.536998 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.536871    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" event={"ID":"dafb9848-08ee-4dac-8004-fe8b2cd747b2","Type":"ContainerDied","Data":"e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7"}
Jul 21 12:57:02.537291 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.537275    2577 scope.go:117] "RemoveContainer" containerID="e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7"
Jul 21 12:57:02.538284 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.538254    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:57:02.538376 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.538325    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:57:02.538376 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:02.538359    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:57:03.543291 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:03.543258    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" event={"ID":"dafb9848-08ee-4dac-8004-fe8b2cd747b2","Type":"ContainerStarted","Data":"329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797"}
Jul 21 12:57:04.049477 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:04.049432    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:57:04.049705 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:04.049680    2577 patch_prober.go:28] interesting pod/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 21 12:57:04.049795 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:04.049771    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 21 12:57:05.050094 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:05.050055    2577 patch_prober.go:28] interesting pod/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 21 12:57:05.050690 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:05.050642    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 21 12:57:06.049997 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:06.049940    2577 patch_prober.go:28] interesting pod/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 21 12:57:06.050174 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:06.050014    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 21 12:57:07.049145 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:07.049107    2577 patch_prober.go:28] interesting pod/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Jul 21 12:57:07.049318 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:07.049174    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Jul 21 12:57:08.053636 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:08.053606    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:57:08.054112 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:08.053866    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:57:08.054642 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:08.054626    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:57:27.953205 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:27.953170    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"]
Jul 21 12:57:27.953694 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:27.953427    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy" containerID="cri-o://329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797" gracePeriod=30
Jul 21 12:57:33.210759 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.210735    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:57:33.409139 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409105    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-credential-socket\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409154    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istiod-ca-cert\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409178    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-socket\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-data\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409234    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-token\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409276    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-envoy\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409328 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409292    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-podinfo\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409640 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409315    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-certs\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409640 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409422    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6w65\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-kube-api-access-v6w65\") pod \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\" (UID: \"dafb9848-08ee-4dac-8004-fe8b2cd747b2\") "
Jul 21 12:57:33.409640 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409419    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:57:33.409640 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409573    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:57:33.409640 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409606    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:57:33.409864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409649    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:57:33.409864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409772    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.409864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409790    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.409864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409803    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.409864 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.409815    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.410300 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.410269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-data" (OuterVolumeSpecName: "istio-data") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:57:33.411460 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.411426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:57:33.411557 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.411461    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:57:33.411605 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.411582    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-token" (OuterVolumeSpecName: "istio-token") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:57:33.411605 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.411588    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-kube-api-access-v6w65" (OuterVolumeSpecName: "kube-api-access-v6w65") pod "dafb9848-08ee-4dac-8004-fe8b2cd747b2" (UID: "dafb9848-08ee-4dac-8004-fe8b2cd747b2"). InnerVolumeSpecName "kube-api-access-v6w65". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:57:33.511092 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.511060    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.511092 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.511088    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.511092 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.511097    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.511324 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.511105    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dafb9848-08ee-4dac-8004-fe8b2cd747b2-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.511324 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.511113    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6w65\" (UniqueName: \"kubernetes.io/projected/dafb9848-08ee-4dac-8004-fe8b2cd747b2-kube-api-access-v6w65\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:57:33.681612 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.681526    2577 generic.go:356] "Generic (PLEG): container finished" podID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerID="329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797" exitCode=0
Jul 21 12:57:33.681612 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.681573    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"
Jul 21 12:57:33.681798 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.681610    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" event={"ID":"dafb9848-08ee-4dac-8004-fe8b2cd747b2","Type":"ContainerDied","Data":"329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797"}
Jul 21 12:57:33.681798 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.681685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw" event={"ID":"dafb9848-08ee-4dac-8004-fe8b2cd747b2","Type":"ContainerDied","Data":"54aa4994cc6f9801d5fae328ddbffc2be2ca4d18da54edabeffed8d754b9717d"}
Jul 21 12:57:33.681798 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.681706    2577 scope.go:117] "RemoveContainer" containerID="329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797"
Jul 21 12:57:33.693368 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.693345    2577 scope.go:117] "RemoveContainer" containerID="e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7"
Jul 21 12:57:33.700935 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.700920    2577 scope.go:117] "RemoveContainer" containerID="329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797"
Jul 21 12:57:33.701253 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:57:33.701192    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797\": container with ID starting with 329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797 not found: ID does not exist" containerID="329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797"
Jul 21 12:57:33.701253 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.701223    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797"} err="failed to get container status \"329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797\": rpc error: code = NotFound desc = could not find container \"329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797\": container with ID starting with 329566d6bc217f60076f8db51e3032e94c2dc5e6822b3b88ff76454640098797 not found: ID does not exist"
Jul 21 12:57:33.701253 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.701244    2577 scope.go:117] "RemoveContainer" containerID="e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7"
Jul 21 12:57:33.701791 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:57:33.701717    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7\": container with ID starting with e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7 not found: ID does not exist" containerID="e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7"
Jul 21 12:57:33.701791 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.701776    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7"} err="failed to get container status \"e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7\": rpc error: code = NotFound desc = could not find container \"e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7\": container with ID starting with e624707ad9dc6abcf18383210a2e4eb86c2deef36db33751ec548c45fb35a6c7 not found: ID does not exist"
Jul 21 12:57:33.703388 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.703367    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"]
Jul 21 12:57:33.706756 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:33.706739    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-6b5db8f8-openshift-default-6587b84947-lkxmw"]
Jul 21 12:57:34.833158 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:57:34.833088    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" path="/var/lib/kubelet/pods/dafb9848-08ee-4dac-8004-fe8b2cd747b2/volumes"
Jul 21 12:59:09.645797 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.645725    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"]
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646090    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646102    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646123    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646129    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646145    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646151    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb83e198-abb2-41ee-b8f9-41fd5cf8139f" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646158    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646163    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646211    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646219    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy"
Jul 21 12:59:09.646271 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.646226    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b114d5a1-53a1-4ca1-a7b1-12990b1e5251" containerName="istio-proxy"
Jul 21 12:59:09.649009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.648992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.651464 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.651438    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-464cd24a-openshift-default-dockercfg-qxlg5\""
Jul 21 12:59:09.672342 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.672307    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"]
Jul 21 12:59:09.700404 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-envoy\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-token\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqlpr\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-kube-api-access-qqlpr\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-podinfo\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700600 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istiod-ca-cert\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700786 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-certs\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700786 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-data\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700786 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-socket\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.700786 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.700726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-credential-socket\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.801491 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.801457    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-envoy\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.801683 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.801499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-token\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.801683 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.801523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqlpr\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-kube-api-access-qqlpr\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.801683 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.801557    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-podinfo\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.801683 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.801599    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istiod-ca-cert\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.801683 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.801637    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-certs\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.801683 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.801680    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-data\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.802065 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.802010    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-socket\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.802065 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.802028    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-certs\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.802065 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.802061    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-credential-socket\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.802233 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.802103    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-data\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.802371 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.802345    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-socket\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.802476 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.802371    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-credential-socket\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.802476 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.802412    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istiod-ca-cert\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.803860 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.803831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-envoy\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.804658 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.804629    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-podinfo\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.810321 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.810301    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-token\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.810413 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.810372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqlpr\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-kube-api-access-qqlpr\") pod \"e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") " pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:09.959205 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:09.959107    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:10.107930 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:10.107906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"]
Jul 21 12:59:10.110545 ip-10-0-143-161 kubenswrapper[2577]: W0721 12:59:10.110518    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2768ff17_f4a9_4ca5_a4ec_abdcee0d6e40.slice/crio-54a4ecffad8731b605845ed13ac7ba044b768a138ece441bf1b237b3b07d5ca4 WatchSource:0}: Error finding container 54a4ecffad8731b605845ed13ac7ba044b768a138ece441bf1b237b3b07d5ca4: Status 404 returned error can't find the container with id 54a4ecffad8731b605845ed13ac7ba044b768a138ece441bf1b237b3b07d5ca4
Jul 21 12:59:10.113069 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:10.113031    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:59:10.113146 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:10.113114    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:59:10.113184 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:10.113155    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:59:11.103378 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:11.103341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" event={"ID":"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40","Type":"ContainerStarted","Data":"2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927"}
Jul 21 12:59:11.103780 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:11.103396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" event={"ID":"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40","Type":"ContainerStarted","Data":"54a4ecffad8731b605845ed13ac7ba044b768a138ece441bf1b237b3b07d5ca4"}
Jul 21 12:59:11.125031 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:11.124944    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" podStartSLOduration=2.124923653 podStartE2EDuration="2.124923653s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 12:59:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 12:59:11.122358297 +0000 UTC m=+1888.907373291" watchObservedRunningTime="2026-07-21 12:59:11.124923653 +0000 UTC m=+1888.909938620"
Jul 21 12:59:11.960031 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:11.959993    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:11.964813 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:11.964780    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:12.107968 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:12.107923    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:12.108935 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:12.108915    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:25.174982 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:25.174929    2577 generic.go:356] "Generic (PLEG): container finished" podID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerID="2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927" exitCode=0
Jul 21 12:59:25.175425 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:25.174995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" event={"ID":"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40","Type":"ContainerDied","Data":"2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927"}
Jul 21 12:59:25.175425 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:25.175403    2577 scope.go:117] "RemoveContainer" containerID="2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927"
Jul 21 12:59:25.176406 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:25.176376    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:59:25.176515 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:25.176443    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:59:25.176515 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:25.176476    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 12:59:26.181486 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:26.181451    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" event={"ID":"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40","Type":"ContainerStarted","Data":"b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d"}
Jul 21 12:59:26.959454 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:26.959416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:26.960544 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:26.960518    2577 patch_prober.go:28] interesting pod/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:59:26.960668 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:26.960577    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:59:27.108116 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:27.108085    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:27.960009 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:27.959946    2577 patch_prober.go:28] interesting pod/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:59:27.960409 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:27.960032    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:59:28.959924 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:28.959889    2577 patch_prober.go:28] interesting pod/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:59:28.960128 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:28.959948    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:59:29.960050 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:29.960010    2577 patch_prober.go:28] interesting pod/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 21 12:59:29.960233 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:29.960065    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 21 12:59:30.963950 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:30.963917    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:30.965087 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:30.965067    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:46.485834 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:46.485803    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"]
Jul 21 12:59:46.486388 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:46.486040    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy" containerID="cri-o://b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d" gracePeriod=30
Jul 21 12:59:51.740949 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.740921    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:51.768684 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.768619    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-certs\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.768684 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.768671    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istiod-ca-cert\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.768876 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.768719    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-socket\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.768876 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.768750    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqlpr\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-kube-api-access-qqlpr\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.768876 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.768772    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-token\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.769057 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.768908    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:59:51.769057 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769023    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-credential-socket\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.769169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769078    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-envoy\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.769169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769085    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:59:51.769169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769115    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-podinfo\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.769169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769129    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 12:59:51.769169 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769156    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-data\") pod \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\" (UID: \"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40\") "
Jul 21 12:59:51.769391 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769275    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:59:51.769588 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769567    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.769701 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769688    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.769790 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769778    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.769871 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.769860    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.770709 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.770674    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-data" (OuterVolumeSpecName: "istio-data") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:59:51.771279 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.771255    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-token" (OuterVolumeSpecName: "istio-token") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:59:51.771636 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.771619    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 12:59:51.771636 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.771625    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-kube-api-access-qqlpr" (OuterVolumeSpecName: "kube-api-access-qqlpr") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "kube-api-access-qqlpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 12:59:51.771852 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.771836    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" (UID: "2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 12:59:51.870778 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.870755    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqlpr\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-kube-api-access-qqlpr\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.870778 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.870776    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.870926 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.870786    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.870926 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.870794    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:51.870926 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:51.870801    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 12:59:52.297473 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.297435    2577 generic.go:356] "Generic (PLEG): container finished" podID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerID="b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d" exitCode=0
Jul 21 12:59:52.297658 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.297485    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"
Jul 21 12:59:52.297658 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.297521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" event={"ID":"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40","Type":"ContainerDied","Data":"b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d"}
Jul 21 12:59:52.297658 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.297577    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl" event={"ID":"2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40","Type":"ContainerDied","Data":"54a4ecffad8731b605845ed13ac7ba044b768a138ece441bf1b237b3b07d5ca4"}
Jul 21 12:59:52.297658 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.297600    2577 scope.go:117] "RemoveContainer" containerID="b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d"
Jul 21 12:59:52.306553 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.306537    2577 scope.go:117] "RemoveContainer" containerID="2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927"
Jul 21 12:59:52.314225 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.314209    2577 scope.go:117] "RemoveContainer" containerID="b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d"
Jul 21 12:59:52.314466 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:59:52.314448    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d\": container with ID starting with b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d not found: ID does not exist" containerID="b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d"
Jul 21 12:59:52.314506 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.314475    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d"} err="failed to get container status \"b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d\": rpc error: code = NotFound desc = could not find container \"b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d\": container with ID starting with b3f76f01bf594a86973002abea1667a991d7ab5830bdc5fd04d2cb303acca63d not found: ID does not exist"
Jul 21 12:59:52.314506 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.314494    2577 scope.go:117] "RemoveContainer" containerID="2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927"
Jul 21 12:59:52.314713 ip-10-0-143-161 kubenswrapper[2577]: E0721 12:59:52.314696    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927\": container with ID starting with 2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927 not found: ID does not exist" containerID="2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927"
Jul 21 12:59:52.314748 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.314719    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927"} err="failed to get container status \"2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927\": rpc error: code = NotFound desc = could not find container \"2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927\": container with ID starting with 2431661e6d991506d4e76688bc3526d4ea59655bdf2f24d815575da573fbf927 not found: ID does not exist"
Jul 21 12:59:52.318040 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.318002    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"]
Jul 21 12:59:52.322369 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.322353    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-464cd24a-openshift-default-bb89cdf99-rnnvl"]
Jul 21 12:59:52.833607 ip-10-0-143-161 kubenswrapper[2577]: I0721 12:59:52.833576    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" path="/var/lib/kubelet/pods/2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40/volumes"
Jul 21 13:00:00.148927 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.148895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94"]
Jul 21 13:00:00.149319 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149226    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy"
Jul 21 13:00:00.149319 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149238    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy"
Jul 21 13:00:00.149319 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149253    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy"
Jul 21 13:00:00.149319 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149260    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy"
Jul 21 13:00:00.149319 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149272    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy"
Jul 21 13:00:00.149319 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149277    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy"
Jul 21 13:00:00.149512 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149335    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="dafb9848-08ee-4dac-8004-fe8b2cd747b2" containerName="istio-proxy"
Jul 21 13:00:00.149512 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149343    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy"
Jul 21 13:00:00.149512 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.149349    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2768ff17-f4a9-4ca5-a4ec-abdcee0d6e40" containerName="istio-proxy"
Jul 21 13:00:00.153561 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.153545    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94"
Jul 21 13:00:00.156286 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.156267    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jq8px\""
Jul 21 13:00:00.163117 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.163098    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94"
Jul 21 13:00:00.164008 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.163988    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94"]
Jul 21 13:00:00.299585 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.297682    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94"]
Jul 21 13:00:00.300702 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:00:00.300660    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0303ce0_8657_4e8d_9ac7_1f40ca66df53.slice/crio-0cb35d492260aa837611c96b96be09b315a4b088136eef5931f87effb49ec8d4 WatchSource:0}: Error finding container 0cb35d492260aa837611c96b96be09b315a4b088136eef5931f87effb49ec8d4: Status 404 returned error can't find the container with id 0cb35d492260aa837611c96b96be09b315a4b088136eef5931f87effb49ec8d4
Jul 21 13:00:00.303199 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.303178    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:00:00.335921 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:00.335880    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94" event={"ID":"f0303ce0-8657-4e8d-9ac7-1f40ca66df53","Type":"ContainerStarted","Data":"0cb35d492260aa837611c96b96be09b315a4b088136eef5931f87effb49ec8d4"}
Jul 21 13:00:06.743771 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.743732    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"]
Jul 21 13:00:06.748307 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.748290    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.750616 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.750589    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-dd701081-openshift-default-dockercfg-zd6cv\""
Jul 21 13:00:06.758645 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.758619    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"]
Jul 21 13:00:06.893561 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dswk7\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-kube-api-access-dswk7\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893733 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-credential-socket\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893733 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893642    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c98342-e575-4df9-947e-b0270441b233-istiod-ca-cert\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893733 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-socket\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893733 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893701    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-istio-token\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893733 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-data\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893897 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893753    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-certs\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893897 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c98342-e575-4df9-947e-b0270441b233-istio-podinfo\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.893897 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.893848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-envoy\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.994521 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-socket\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.994521 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994481    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-istio-token\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.994521 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994504    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-data\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.994798 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-certs\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.994798 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994702    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c98342-e575-4df9-947e-b0270441b233-istio-podinfo\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.994798 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-envoy\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995050 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dswk7\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-kube-api-access-dswk7\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995050 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-credential-socket\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995050 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994887    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-certs\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995050 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994900    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c98342-e575-4df9-947e-b0270441b233-istiod-ca-cert\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995050 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994910    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-socket\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995050 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.994985    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-data\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995304 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.995230    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-credential-socket\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.995606 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.995585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c98342-e575-4df9-947e-b0270441b233-istiod-ca-cert\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.997276 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.997245    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c98342-e575-4df9-947e-b0270441b233-istio-podinfo\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:06.997373 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:06.997344    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-envoy\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:07.002876 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.002841    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dswk7\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-kube-api-access-dswk7\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:07.003285 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.003270    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-istio-token\") pod \"e2e-mt-dd701081-openshift-default-78545755f5-5nqvn\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") " pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:07.060434 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.060405    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:07.194678 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.194652    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"]
Jul 21 13:00:07.196836 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:00:07.196811    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10c98342_e575_4df9_947e_b0270441b233.slice/crio-d98262f5b123c1cde8b1cefd5e146917df3945c485458157371bce358c859f72 WatchSource:0}: Error finding container d98262f5b123c1cde8b1cefd5e146917df3945c485458157371bce358c859f72: Status 404 returned error can't find the container with id d98262f5b123c1cde8b1cefd5e146917df3945c485458157371bce358c859f72
Jul 21 13:00:07.199409 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.199370    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:00:07.199516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.199439    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:00:07.199516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.199470    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:00:07.369487 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.369443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" event={"ID":"10c98342-e575-4df9-947e-b0270441b233","Type":"ContainerStarted","Data":"5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b"}
Jul 21 13:00:07.369676 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.369516    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" event={"ID":"10c98342-e575-4df9-947e-b0270441b233","Type":"ContainerStarted","Data":"d98262f5b123c1cde8b1cefd5e146917df3945c485458157371bce358c859f72"}
Jul 21 13:00:07.391929 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:07.391876    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podStartSLOduration=1.391855301 podStartE2EDuration="1.391855301s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:07.389413307 +0000 UTC m=+1945.174428307" watchObservedRunningTime="2026-07-21 13:00:07.391855301 +0000 UTC m=+1945.176870268"
Jul 21 13:00:08.060832 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:08.060805    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:08.065655 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:08.065632    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:08.374982 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:08.374871    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:08.375680 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:08.375660    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:20.432543 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:20.432499    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94" event={"ID":"f0303ce0-8657-4e8d-9ac7-1f40ca66df53","Type":"ContainerStarted","Data":"04fe58f3316133285693a67545c00582b9ae8383fdf4388d250d86c763df7726"}
Jul 21 13:00:20.448456 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:20.448397    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94" podStartSLOduration=1.293540441 podStartE2EDuration="20.448377375s" totalImagesPullingTime="19.154836934s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:20.44775698 +0000 UTC m=+1958.232771948" watchObservedRunningTime="2026-07-21 13:00:20.448377375 +0000 UTC m=+1958.233392341"
Jul 21 13:00:21.441443 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:21.441404    2577 generic.go:356] "Generic (PLEG): container finished" podID="f0303ce0-8657-4e8d-9ac7-1f40ca66df53" containerID="04fe58f3316133285693a67545c00582b9ae8383fdf4388d250d86c763df7726" exitCode=0
Jul 21 13:00:21.441926 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:21.441535    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94" event={"ID":"f0303ce0-8657-4e8d-9ac7-1f40ca66df53","Type":"ContainerDied","Data":"04fe58f3316133285693a67545c00582b9ae8383fdf4388d250d86c763df7726"}
Jul 21 13:00:22.588849 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:22.588823    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94"
Jul 21 13:00:23.453156 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:23.453117    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94"
Jul 21 13:00:23.453156 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:23.453145    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743980-tcv94" event={"ID":"f0303ce0-8657-4e8d-9ac7-1f40ca66df53","Type":"ContainerDied","Data":"0cb35d492260aa837611c96b96be09b315a4b088136eef5931f87effb49ec8d4"}
Jul 21 13:00:23.453373 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:23.453202    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cb35d492260aa837611c96b96be09b315a4b088136eef5931f87effb49ec8d4"
Jul 21 13:00:24.416139 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:24.416099    2577 patch_prober.go:28] interesting pod/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 21 13:00:24.417780 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:24.416168    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 21 13:00:25.464576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:25.464540    2577 generic.go:356] "Generic (PLEG): container finished" podID="10c98342-e575-4df9-947e-b0270441b233" containerID="5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b" exitCode=0
Jul 21 13:00:25.465064 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:25.464592    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" event={"ID":"10c98342-e575-4df9-947e-b0270441b233","Type":"ContainerDied","Data":"5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b"}
Jul 21 13:00:25.465136 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:25.465094    2577 scope.go:117] "RemoveContainer" containerID="5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b"
Jul 21 13:00:25.466095 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:25.466064    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:00:25.466197 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:25.466133    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:00:25.466197 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:25.466166    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:00:26.470601 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:26.470569    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" event={"ID":"10c98342-e575-4df9-947e-b0270441b233","Type":"ContainerStarted","Data":"00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb"}
Jul 21 13:00:27.061494 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:27.061458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:27.061752 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:27.061728    2577 patch_prober.go:28] interesting pod/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 13:00:27.061797 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:27.061778    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 13:00:28.061336 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:28.061293    2577 patch_prober.go:28] interesting pod/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 13:00:28.061714 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:28.061356    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 13:00:29.061338 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:29.061301    2577 patch_prober.go:28] interesting pod/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 13:00:29.061732 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:29.061362    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 13:00:30.061230 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:30.061192    2577 patch_prober.go:28] interesting pod/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body=
Jul 21 13:00:30.061451 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:30.061248    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused"
Jul 21 13:00:31.064485 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:31.064452    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:31.064918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:31.064717    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:31.065462 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:31.065443    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:00:40.988099 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.988064    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"]
Jul 21 13:00:40.988455 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.988413    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0303ce0-8657-4e8d-9ac7-1f40ca66df53" containerName="cleanup"
Jul 21 13:00:40.988455 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.988429    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0303ce0-8657-4e8d-9ac7-1f40ca66df53" containerName="cleanup"
Jul 21 13:00:40.988550 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.988519    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0303ce0-8657-4e8d-9ac7-1f40ca66df53" containerName="cleanup"
Jul 21 13:00:40.993247 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.993225    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:40.996449 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.996421    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dd701081\"/\"default-dockercfg-4jhbg\""
Jul 21 13:00:40.996576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.996499    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dd701081\"/\"kube-root-ca.crt\""
Jul 21 13:00:40.996576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.996499    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-dd701081\"/\"e2e-lifecycle-model-dd701081-kserve-self-signed-certs\""
Jul 21 13:00:40.996576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:40.996538    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-dd701081\"/\"openshift-service-ca.crt\""
Jul 21 13:00:41.001804 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.001780    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"]
Jul 21 13:00:41.098519 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.098490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-tmp-dir\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.098691 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.098529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-dshm\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.098691 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.098550    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-model-cache\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.098691 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.098604    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-home\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.098691 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.098636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgpnp\" (UniqueName: \"kubernetes.io/projected/540c7722-115a-49dc-b5ab-0a1462f7a853-kube-api-access-mgpnp\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.098691 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.098683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/540c7722-115a-49dc-b5ab-0a1462f7a853-tls-certs\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.199885 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.199849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-tmp-dir\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200076 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.199894    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-dshm\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200076 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.199912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-model-cache\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200076 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.199946    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-home\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200076 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.199995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgpnp\" (UniqueName: \"kubernetes.io/projected/540c7722-115a-49dc-b5ab-0a1462f7a853-kube-api-access-mgpnp\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200076 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.200056    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/540c7722-115a-49dc-b5ab-0a1462f7a853-tls-certs\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200351 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.200321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-tmp-dir\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200351 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.200346    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-home\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.200473 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.200443    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-model-cache\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.202248 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.202225    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-dshm\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.202478 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.202459    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/540c7722-115a-49dc-b5ab-0a1462f7a853-tls-certs\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.208273 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.208250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgpnp\" (UniqueName: \"kubernetes.io/projected/540c7722-115a-49dc-b5ab-0a1462f7a853-kube-api-access-mgpnp\") pod \"e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") " pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.304093 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.304011    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:41.437841 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.437815    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"]
Jul 21 13:00:41.439740 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:00:41.439707    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod540c7722_115a_49dc_b5ab_0a1462f7a853.slice/crio-e900ab3a9e036ecc8403dfbb41e98867d0592a74fb1a53df22ebcdd5420ecc17 WatchSource:0}: Error finding container e900ab3a9e036ecc8403dfbb41e98867d0592a74fb1a53df22ebcdd5420ecc17: Status 404 returned error can't find the container with id e900ab3a9e036ecc8403dfbb41e98867d0592a74fb1a53df22ebcdd5420ecc17
Jul 21 13:00:41.546414 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.546375    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4" event={"ID":"540c7722-115a-49dc-b5ab-0a1462f7a853","Type":"ContainerStarted","Data":"8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61"}
Jul 21 13:00:41.546573 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.546422    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4" event={"ID":"540c7722-115a-49dc-b5ab-0a1462f7a853","Type":"ContainerStarted","Data":"e900ab3a9e036ecc8403dfbb41e98867d0592a74fb1a53df22ebcdd5420ecc17"}
Jul 21 13:00:41.565050 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:41.564937    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4" podStartSLOduration=1.5649220480000001 podStartE2EDuration="1.564922048s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:00:41.563184351 +0000 UTC m=+1979.348199318" watchObservedRunningTime="2026-07-21 13:00:41.564922048 +0000 UTC m=+1979.349937013"
Jul 21 13:00:51.304603 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:51.304566    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:51.305020 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:51.304613    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:51.317331 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:51.317307    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:00:51.607472 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:00:51.607391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:01:30.252498 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:30.252458    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"]
Jul 21 13:01:45.272226 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.272186    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4" podUID="540c7722-115a-49dc-b5ab-0a1462f7a853" containerName="main" containerID="cri-o://8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61" gracePeriod=45
Jul 21 13:01:45.518500 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.518475    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:01:45.537187 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537119    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/540c7722-115a-49dc-b5ab-0a1462f7a853-tls-certs\") pod \"540c7722-115a-49dc-b5ab-0a1462f7a853\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") "
Jul 21 13:01:45.537313 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537197    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-model-cache\") pod \"540c7722-115a-49dc-b5ab-0a1462f7a853\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") "
Jul 21 13:01:45.537313 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537223    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-home\") pod \"540c7722-115a-49dc-b5ab-0a1462f7a853\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") "
Jul 21 13:01:45.537313 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537262    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgpnp\" (UniqueName: \"kubernetes.io/projected/540c7722-115a-49dc-b5ab-0a1462f7a853-kube-api-access-mgpnp\") pod \"540c7722-115a-49dc-b5ab-0a1462f7a853\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") "
Jul 21 13:01:45.537313 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537305    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-tmp-dir\") pod \"540c7722-115a-49dc-b5ab-0a1462f7a853\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") "
Jul 21 13:01:45.537549 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537322    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-dshm\") pod \"540c7722-115a-49dc-b5ab-0a1462f7a853\" (UID: \"540c7722-115a-49dc-b5ab-0a1462f7a853\") "
Jul 21 13:01:45.537549 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537479    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-model-cache" (OuterVolumeSpecName: "model-cache") pod "540c7722-115a-49dc-b5ab-0a1462f7a853" (UID: "540c7722-115a-49dc-b5ab-0a1462f7a853"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:45.537652 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537496    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-home" (OuterVolumeSpecName: "home") pod "540c7722-115a-49dc-b5ab-0a1462f7a853" (UID: "540c7722-115a-49dc-b5ab-0a1462f7a853"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:45.537652 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.537588    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "540c7722-115a-49dc-b5ab-0a1462f7a853" (UID: "540c7722-115a-49dc-b5ab-0a1462f7a853"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:45.539596 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.539556    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/540c7722-115a-49dc-b5ab-0a1462f7a853-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "540c7722-115a-49dc-b5ab-0a1462f7a853" (UID: "540c7722-115a-49dc-b5ab-0a1462f7a853"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:01:45.539596 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.539578    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/540c7722-115a-49dc-b5ab-0a1462f7a853-kube-api-access-mgpnp" (OuterVolumeSpecName: "kube-api-access-mgpnp") pod "540c7722-115a-49dc-b5ab-0a1462f7a853" (UID: "540c7722-115a-49dc-b5ab-0a1462f7a853"). InnerVolumeSpecName "kube-api-access-mgpnp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:01:45.539794 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.539619    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-dshm" (OuterVolumeSpecName: "dshm") pod "540c7722-115a-49dc-b5ab-0a1462f7a853" (UID: "540c7722-115a-49dc-b5ab-0a1462f7a853"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:01:45.638483 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.638447    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgpnp\" (UniqueName: \"kubernetes.io/projected/540c7722-115a-49dc-b5ab-0a1462f7a853-kube-api-access-mgpnp\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:01:45.638483 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.638476    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-tmp-dir\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:01:45.638483 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.638489    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-dshm\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:01:45.638707 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.638496    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/540c7722-115a-49dc-b5ab-0a1462f7a853-tls-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:01:45.638707 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.638504    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-model-cache\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:01:45.638707 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.638510    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/540c7722-115a-49dc-b5ab-0a1462f7a853-home\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:01:45.842086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.841986    2577 generic.go:356] "Generic (PLEG): container finished" podID="540c7722-115a-49dc-b5ab-0a1462f7a853" containerID="8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61" exitCode=0
Jul 21 13:01:45.842086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.842034    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"
Jul 21 13:01:45.842322 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.842078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4" event={"ID":"540c7722-115a-49dc-b5ab-0a1462f7a853","Type":"ContainerDied","Data":"8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61"}
Jul 21 13:01:45.842322 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.842121    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4" event={"ID":"540c7722-115a-49dc-b5ab-0a1462f7a853","Type":"ContainerDied","Data":"e900ab3a9e036ecc8403dfbb41e98867d0592a74fb1a53df22ebcdd5420ecc17"}
Jul 21 13:01:45.842322 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.842143    2577 scope.go:117] "RemoveContainer" containerID="8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61"
Jul 21 13:01:45.851115 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.851098    2577 scope.go:117] "RemoveContainer" containerID="8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61"
Jul 21 13:01:45.851340 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:01:45.851323    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61\": container with ID starting with 8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61 not found: ID does not exist" containerID="8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61"
Jul 21 13:01:45.851388 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.851346    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61"} err="failed to get container status \"8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61\": rpc error: code = NotFound desc = could not find container \"8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61\": container with ID starting with 8ecc115808e8eb7f646e0be16fc09cc4f0ac78ec341d472ba98af151e1108f61 not found: ID does not exist"
Jul 21 13:01:45.863698 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.863679    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"]
Jul 21 13:01:45.866466 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:45.866446    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-dd701081/e2e-lifecycle-model-dd701081-kserve-655ff4dcff-k7qz4"]
Jul 21 13:01:46.832679 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:01:46.832646    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="540c7722-115a-49dc-b5ab-0a1462f7a853" path="/var/lib/kubelet/pods/540c7722-115a-49dc-b5ab-0a1462f7a853/volumes"
Jul 21 13:02:14.114247 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:14.114164    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"]
Jul 21 13:02:14.114687 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:14.114376    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy" containerID="cri-o://00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb" gracePeriod=30
Jul 21 13:02:16.067060 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:16.067010    2577 patch_prober.go:28] interesting pod/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 13:02:16.067435 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:16.067100    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 13:02:19.363176 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.363153    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:02:19.526120 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526090    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-data\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526312 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526135    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-certs\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526312 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526169    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dswk7\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-kube-api-access-dswk7\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526312 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526194    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c98342-e575-4df9-947e-b0270441b233-istiod-ca-cert\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526312 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526245    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-istio-token\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526312 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526304    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-envoy\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526585 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526344    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-credential-socket\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526585 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526384    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-socket\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526585 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c98342-e575-4df9-947e-b0270441b233-istio-podinfo\") pod \"10c98342-e575-4df9-947e-b0270441b233\" (UID: \"10c98342-e575-4df9-947e-b0270441b233\") "
Jul 21 13:02:19.526585 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526445    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:19.526838 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526674    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.526838 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526685    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:19.526838 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.526720    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/10c98342-e575-4df9-947e-b0270441b233-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:02:19.527225 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.527184    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:19.527682 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.527657    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-data" (OuterVolumeSpecName: "istio-data") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:19.528794 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.528761    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-istio-token" (OuterVolumeSpecName: "istio-token") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:02:19.528794 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.528778    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-kube-api-access-dswk7" (OuterVolumeSpecName: "kube-api-access-dswk7") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "kube-api-access-dswk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:02:19.528931 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.528806    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/10c98342-e575-4df9-947e-b0270441b233-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:02:19.528931 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.528897    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "10c98342-e575-4df9-947e-b0270441b233" (UID: "10c98342-e575-4df9-947e-b0270441b233"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:02:19.627077 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627052    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.627077 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627076    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dswk7\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-kube-api-access-dswk7\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.627218 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627086    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/10c98342-e575-4df9-947e-b0270441b233-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.627218 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627096    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/10c98342-e575-4df9-947e-b0270441b233-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.627218 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627104    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.627218 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627111    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.627218 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627118    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/10c98342-e575-4df9-947e-b0270441b233-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.627218 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.627126    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/10c98342-e575-4df9-947e-b0270441b233-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:02:19.996425 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.996396    2577 generic.go:356] "Generic (PLEG): container finished" podID="10c98342-e575-4df9-947e-b0270441b233" containerID="00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb" exitCode=0
Jul 21 13:02:19.996582 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.996446    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" event={"ID":"10c98342-e575-4df9-947e-b0270441b233","Type":"ContainerDied","Data":"00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb"}
Jul 21 13:02:19.996582 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.996482    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn" event={"ID":"10c98342-e575-4df9-947e-b0270441b233","Type":"ContainerDied","Data":"d98262f5b123c1cde8b1cefd5e146917df3945c485458157371bce358c859f72"}
Jul 21 13:02:19.996582 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.996504    2577 scope.go:117] "RemoveContainer" containerID="00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb"
Jul 21 13:02:19.996582 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:19.996446    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"
Jul 21 13:02:20.011723 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.011705    2577 scope.go:117] "RemoveContainer" containerID="5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b"
Jul 21 13:02:20.019365 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.019348    2577 scope.go:117] "RemoveContainer" containerID="00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb"
Jul 21 13:02:20.019618 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:02:20.019599    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb\": container with ID starting with 00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb not found: ID does not exist" containerID="00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb"
Jul 21 13:02:20.019688 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.019630    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb"} err="failed to get container status \"00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb\": rpc error: code = NotFound desc = could not find container \"00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb\": container with ID starting with 00dddd88d41aa429b2a3df968c6d9d50ecc07e7afeacfbe4c9b2182ed83b02eb not found: ID does not exist"
Jul 21 13:02:20.019688 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.019655    2577 scope.go:117] "RemoveContainer" containerID="5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b"
Jul 21 13:02:20.019885 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:02:20.019867    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b\": container with ID starting with 5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b not found: ID does not exist" containerID="5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b"
Jul 21 13:02:20.019922 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.019891    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b"} err="failed to get container status \"5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b\": rpc error: code = NotFound desc = could not find container \"5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b\": container with ID starting with 5284ef99b0d911531df5ead33333d760ee322adfaf359712e809ad235f28432b not found: ID does not exist"
Jul 21 13:02:20.026359 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.026336    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"]
Jul 21 13:02:20.031431 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.031409    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-dd701081-openshift-default-78545755f5-5nqvn"]
Jul 21 13:02:20.833319 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:02:20.833283    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10c98342-e575-4df9-947e-b0270441b233" path="/var/lib/kubelet/pods/10c98342-e575-4df9-947e-b0270441b233/volumes"
Jul 21 13:03:10.265393 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265357    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"]
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265805    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265824    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265850    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265858    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265886    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="540c7722-115a-49dc-b5ab-0a1462f7a853" containerName="main"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265894    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="540c7722-115a-49dc-b5ab-0a1462f7a853" containerName="main"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.265993    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="540c7722-115a-49dc-b5ab-0a1462f7a853" containerName="main"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.266007    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy"
Jul 21 13:03:10.266011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.266016    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="10c98342-e575-4df9-947e-b0270441b233" containerName="istio-proxy"
Jul 21 13:03:10.269130 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.269112    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.272623 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.272599    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-52a2ba-openshift-default-dockercfg-nqmp8\""
Jul 21 13:03:10.289581 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.289554    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"]
Jul 21 13:03:10.362660 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362620    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-credential-socket\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-data\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362699    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-envoy\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-podinfo\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-certs\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362798    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istiod-ca-cert\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362816    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-token\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-socket\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.362874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.362873    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z26nh\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-kube-api-access-z26nh\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.463735 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.463699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-envoy\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.463933 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.463753    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-podinfo\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.463933 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.463801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-certs\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.463933 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.463817    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istiod-ca-cert\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.463933 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.463837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-token\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464193 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464095    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-socket\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464290 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464249    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z26nh\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-kube-api-access-z26nh\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464354 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464323    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-certs\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464412 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464349    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-credential-socket\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464468 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-data\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464529 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464419    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-socket\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464669 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464644    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-credential-socket\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464736 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464669    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-data\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.464736 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.464679    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istiod-ca-cert\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.466491 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.466470    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-envoy\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.466622 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.466605    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-podinfo\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.472515 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.472488    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-token\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.472647 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.472629    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z26nh\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-kube-api-access-z26nh\") pod \"e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") " pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.579098 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.578982    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:10.723467 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.723351    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"]
Jul 21 13:03:10.726368 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:03:10.726328    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cc0cd7b_07ae_4621_a54a_abdc5070863d.slice/crio-59267ad460bd4e0db4f970d6f1db2d0db7cf982a905ce0e9f414bc1cf54baee3 WatchSource:0}: Error finding container 59267ad460bd4e0db4f970d6f1db2d0db7cf982a905ce0e9f414bc1cf54baee3: Status 404 returned error can't find the container with id 59267ad460bd4e0db4f970d6f1db2d0db7cf982a905ce0e9f414bc1cf54baee3
Jul 21 13:03:10.729310 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.729270    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:10.729415 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.729362    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:10.729415 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:10.729400    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:11.229398 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:11.229364    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" event={"ID":"9cc0cd7b-07ae-4621-a54a-abdc5070863d","Type":"ContainerStarted","Data":"fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5"}
Jul 21 13:03:11.229587 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:11.229417    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" event={"ID":"9cc0cd7b-07ae-4621-a54a-abdc5070863d","Type":"ContainerStarted","Data":"59267ad460bd4e0db4f970d6f1db2d0db7cf982a905ce0e9f414bc1cf54baee3"}
Jul 21 13:03:11.250453 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:11.250399    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" podStartSLOduration=1.250383271 podStartE2EDuration="1.250383271s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:03:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:03:11.249137772 +0000 UTC m=+2129.034152739" watchObservedRunningTime="2026-07-21 13:03:11.250383271 +0000 UTC m=+2129.035398236"
Jul 21 13:03:11.579901 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:11.579859    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:11.584810 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:11.584783    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:12.234646 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:12.234606    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:12.235681 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:12.235658    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:18.423803 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.423762    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"]
Jul 21 13:03:18.427631 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.427606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.430474 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.430448    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0bdb53-openshift-default-dockercfg-5v9pt\""
Jul 21 13:03:18.433286 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433251    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istiod-ca-cert\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433411 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433299    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-podinfo\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433461 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433407    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-credential-socket\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433461 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-socket\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433555 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-data\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433555 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cmk5s\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-kube-api-access-cmk5s\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433658 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-envoy\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433658 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-certs\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.433768 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.433652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-token\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.439279 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.439252    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"]
Jul 21 13:03:18.534533 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.534467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-credential-socket\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.534533 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.534512    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-socket\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.534823 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.534543    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-data\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.534823 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.534568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cmk5s\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-kube-api-access-cmk5s\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.534975 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.534933    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-credential-socket\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535136 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535106    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-data\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535136 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-envoy\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535296 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535148    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-socket\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535296 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535167    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-certs\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535296 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-token\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535296 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istiod-ca-cert\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535296 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535271    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-podinfo\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.535533 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.535466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-certs\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.536182 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.536144    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istiod-ca-cert\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.537587 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.537560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-envoy\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.538020 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.538000    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-podinfo\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.543652 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.543573    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-token\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.544041 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.544019    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cmk5s\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-kube-api-access-cmk5s\") pod \"e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") " pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.742466 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.742424    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:18.893617 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.893592    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"]
Jul 21 13:03:18.895499 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:03:18.895470    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8705b482_3677_4556_b25f_8ab6ce3d8bb1.slice/crio-7ca044a539068b35876e5f8eb934e42c700afa34459a52f77a265f57d4b2aa2c WatchSource:0}: Error finding container 7ca044a539068b35876e5f8eb934e42c700afa34459a52f77a265f57d4b2aa2c: Status 404 returned error can't find the container with id 7ca044a539068b35876e5f8eb934e42c700afa34459a52f77a265f57d4b2aa2c
Jul 21 13:03:18.897598 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.897564    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:18.897704 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.897647    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:18.897704 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:18.897695    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:19.268709 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:19.268672    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" event={"ID":"8705b482-3677-4556-b25f-8ab6ce3d8bb1","Type":"ContainerStarted","Data":"bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197"}
Jul 21 13:03:19.268890 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:19.268769    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" event={"ID":"8705b482-3677-4556-b25f-8ab6ce3d8bb1","Type":"ContainerStarted","Data":"7ca044a539068b35876e5f8eb934e42c700afa34459a52f77a265f57d4b2aa2c"}
Jul 21 13:03:19.290875 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:19.290812    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" podStartSLOduration=1.290789489 podStartE2EDuration="1.290789489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:03:19.289297982 +0000 UTC m=+2137.074312947" watchObservedRunningTime="2026-07-21 13:03:19.290789489 +0000 UTC m=+2137.075804456"
Jul 21 13:03:19.745056 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:19.745012    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:19.753094 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:19.753057    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:20.273713 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:20.273685    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:20.274756 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:20.274735    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:27.236285 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.236242    2577 patch_prober.go:28] interesting pod/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 13:03:27.237287 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.236945    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 13:03:27.311367 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.311332    2577 generic.go:356] "Generic (PLEG): container finished" podID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerID="fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5" exitCode=0
Jul 21 13:03:27.311540 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.311408    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" event={"ID":"9cc0cd7b-07ae-4621-a54a-abdc5070863d","Type":"ContainerDied","Data":"fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5"}
Jul 21 13:03:27.311844 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.311819    2577 scope.go:117] "RemoveContainer" containerID="fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5"
Jul 21 13:03:27.312801 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.312772    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:27.312853 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.312843    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:27.312892 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:27.312878    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:28.319152 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:28.319118    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" event={"ID":"9cc0cd7b-07ae-4621-a54a-abdc5070863d","Type":"ContainerStarted","Data":"bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a"}
Jul 21 13:03:28.580131 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:28.579897    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:28.580410 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:28.580379    2577 patch_prober.go:28] interesting pod/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 13:03:28.580474 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:28.580453    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 13:03:29.579713 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:29.579672    2577 patch_prober.go:28] interesting pod/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 13:03:29.580148 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:29.579725    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 13:03:30.579390 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:30.579352    2577 patch_prober.go:28] interesting pod/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 13:03:30.579586 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:30.579405    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 13:03:31.579584 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:31.579536    2577 patch_prober.go:28] interesting pod/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body=
Jul 21 13:03:31.580119 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:31.579601    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused"
Jul 21 13:03:32.583922 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:32.583891    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:32.584321 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:32.584205    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:32.584868 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:32.584852    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:03:34.354536 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:34.354449    2577 generic.go:356] "Generic (PLEG): container finished" podID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerID="bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197" exitCode=0
Jul 21 13:03:34.354536 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:34.354521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" event={"ID":"8705b482-3677-4556-b25f-8ab6ce3d8bb1","Type":"ContainerDied","Data":"bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197"}
Jul 21 13:03:34.355076 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:34.355055    2577 scope.go:117] "RemoveContainer" containerID="bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197"
Jul 21 13:03:34.356127 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:34.356100    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:34.356200 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:34.356183    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:34.356250 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:34.356234    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:03:35.274291 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:35.274243    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:35.360551 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:35.360510    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" event={"ID":"8705b482-3677-4556-b25f-8ab6ce3d8bb1","Type":"ContainerStarted","Data":"be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee"}
Jul 21 13:03:35.743457 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:35.743403    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:35.744742 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:35.744701    2577 patch_prober.go:28] interesting pod/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 21 13:03:35.744903 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:35.744786    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 21 13:03:36.743118 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:36.743068    2577 patch_prober.go:28] interesting pod/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 21 13:03:36.743632 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:36.743143    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 21 13:03:37.743125 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:37.743078    2577 patch_prober.go:28] interesting pod/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 21 13:03:37.743724 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:37.743692    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 21 13:03:38.743149 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:38.743106    2577 patch_prober.go:28] interesting pod/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Jul 21 13:03:38.743749 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:38.743715    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Jul 21 13:03:39.746454 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:39.746419    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:39.746841 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:39.746797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:03:39.747591 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:03:39.747572    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:04:05.088053 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.088017    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"]
Jul 21 13:04:05.091035 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.091000    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.094001 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.093951    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0bdb53\"/\"default-dockercfg-xt5lv\""
Jul 21 13:04:05.094854 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.094834    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0bdb53\"/\"kube-root-ca.crt\""
Jul 21 13:04:05.094923 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.094863    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0bdb53\"/\"test-model-0bdb53-kserve-self-signed-certs\""
Jul 21 13:04:05.095000 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.094864    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0bdb53\"/\"openshift-service-ca.crt\""
Jul 21 13:04:05.103106 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.103083    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"]
Jul 21 13:04:05.154786 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.154747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-home\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.154997 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.154814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-model-cache\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.154997 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.154846    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-tmp-dir\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.154997 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.154867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfmjw\" (UniqueName: \"kubernetes.io/projected/a898f824-e2f9-4f13-989d-3364ad568cad-kube-api-access-bfmjw\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.154997 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.154981    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-dshm\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.155184 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.155011    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a898f824-e2f9-4f13-989d-3364ad568cad-tls-certs\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.255426 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.255390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfmjw\" (UniqueName: \"kubernetes.io/projected/a898f824-e2f9-4f13-989d-3364ad568cad-kube-api-access-bfmjw\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.255638 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.255470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-dshm\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.255638 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.255503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a898f824-e2f9-4f13-989d-3364ad568cad-tls-certs\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.255638 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.255538    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-home\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.255638 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.255584    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-model-cache\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.255638 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.255612    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-tmp-dir\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.256035 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.256008    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-model-cache\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.256120 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.256045    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-home\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.256120 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.256074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-tmp-dir\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.257918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.257893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-dshm\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.258067 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.258053    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a898f824-e2f9-4f13-989d-3364ad568cad-tls-certs\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.264054 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.264030    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfmjw\" (UniqueName: \"kubernetes.io/projected/a898f824-e2f9-4f13-989d-3364ad568cad-kube-api-access-bfmjw\") pod \"test-model-0bdb53-kserve-5f977d66f7-5bxk8\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") " pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.401378 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.401281    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:05.544992 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:05.544943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"]
Jul 21 13:04:05.547486 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:04:05.547461    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda898f824_e2f9_4f13_989d_3364ad568cad.slice/crio-ca40bf69d5b3bb1591f614b8771bfd6ab6f210bb7c622f8c8815b211e03dcb9b WatchSource:0}: Error finding container ca40bf69d5b3bb1591f614b8771bfd6ab6f210bb7c622f8c8815b211e03dcb9b: Status 404 returned error can't find the container with id ca40bf69d5b3bb1591f614b8771bfd6ab6f210bb7c622f8c8815b211e03dcb9b
Jul 21 13:04:06.512072 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:06.512033    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8" event={"ID":"a898f824-e2f9-4f13-989d-3364ad568cad","Type":"ContainerStarted","Data":"fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f"}
Jul 21 13:04:06.512511 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:06.512084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8" event={"ID":"a898f824-e2f9-4f13-989d-3364ad568cad","Type":"ContainerStarted","Data":"ca40bf69d5b3bb1591f614b8771bfd6ab6f210bb7c622f8c8815b211e03dcb9b"}
Jul 21 13:04:06.531727 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:06.531670    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8" podStartSLOduration=1.53165595 podStartE2EDuration="1.53165595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:04:06.529260991 +0000 UTC m=+2184.314275957" watchObservedRunningTime="2026-07-21 13:04:06.53165595 +0000 UTC m=+2184.316670951"
Jul 21 13:04:15.401628 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:15.401590    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:15.401628 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:15.401634    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:15.414301 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:15.414276    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:15.570780 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:15.570752    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:04:46.462981 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.462934    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"]
Jul 21 13:04:46.604142 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.604101    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"]
Jul 21 13:04:46.606672 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.606650    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"
Jul 21 13:04:46.609105 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.609079    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jq8px\""
Jul 21 13:04:46.609244 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.609079    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:04:46.615471 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.615449    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"]
Jul 21 13:04:46.723387 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.723291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/579d9289-7003-44c2-b208-b7455825a95a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm\" (UID: \"579d9289-7003-44c2-b208-b7455825a95a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"
Jul 21 13:04:46.825032 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.824983    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/579d9289-7003-44c2-b208-b7455825a95a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm\" (UID: \"579d9289-7003-44c2-b208-b7455825a95a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"
Jul 21 13:04:46.825860 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.825836    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/579d9289-7003-44c2-b208-b7455825a95a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm\" (UID: \"579d9289-7003-44c2-b208-b7455825a95a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"
Jul 21 13:04:46.918062 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:46.918019    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"
Jul 21 13:04:47.063122 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:47.063075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"]
Jul 21 13:04:47.070115 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:04:47.070067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579d9289_7003_44c2_b208_b7455825a95a.slice/crio-01adce0f06f0bcec39c55a7024c43faaf9f0e44d1e1bab9deceaea579997182e WatchSource:0}: Error finding container 01adce0f06f0bcec39c55a7024c43faaf9f0e44d1e1bab9deceaea579997182e: Status 404 returned error can't find the container with id 01adce0f06f0bcec39c55a7024c43faaf9f0e44d1e1bab9deceaea579997182e
Jul 21 13:04:47.714654 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:47.714615    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm" event={"ID":"579d9289-7003-44c2-b208-b7455825a95a","Type":"ContainerStarted","Data":"1d3a62067bdc69fb230b824426c63686ee23be1245517a2912fd1b5d6962c353"}
Jul 21 13:04:47.715051 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:47.714679    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm" event={"ID":"579d9289-7003-44c2-b208-b7455825a95a","Type":"ContainerStarted","Data":"01adce0f06f0bcec39c55a7024c43faaf9f0e44d1e1bab9deceaea579997182e"}
Jul 21 13:04:47.732983 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:47.732924    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm" podStartSLOduration=1.732905523 podStartE2EDuration="1.732905523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:04:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:04:47.731444982 +0000 UTC m=+2225.516459947" watchObservedRunningTime="2026-07-21 13:04:47.732905523 +0000 UTC m=+2225.517920492"
Jul 21 13:04:48.721058 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:48.721024    2577 generic.go:356] "Generic (PLEG): container finished" podID="579d9289-7003-44c2-b208-b7455825a95a" containerID="1d3a62067bdc69fb230b824426c63686ee23be1245517a2912fd1b5d6962c353" exitCode=0
Jul 21 13:04:48.721443 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:48.721064    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm" event={"ID":"579d9289-7003-44c2-b208-b7455825a95a","Type":"ContainerDied","Data":"1d3a62067bdc69fb230b824426c63686ee23be1245517a2912fd1b5d6962c353"}
Jul 21 13:04:49.858982 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:49.858933    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"
Jul 21 13:04:49.955696 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:49.955659    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/579d9289-7003-44c2-b208-b7455825a95a-maas-api-service-ca\") pod \"579d9289-7003-44c2-b208-b7455825a95a\" (UID: \"579d9289-7003-44c2-b208-b7455825a95a\") "
Jul 21 13:04:49.956119 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:49.956093    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/579d9289-7003-44c2-b208-b7455825a95a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "579d9289-7003-44c2-b208-b7455825a95a" (UID: "579d9289-7003-44c2-b208-b7455825a95a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:04:50.057436 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:50.057400    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/579d9289-7003-44c2-b208-b7455825a95a-maas-api-service-ca\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:04:50.733034 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:50.732996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm" event={"ID":"579d9289-7003-44c2-b208-b7455825a95a","Type":"ContainerDied","Data":"01adce0f06f0bcec39c55a7024c43faaf9f0e44d1e1bab9deceaea579997182e"}
Jul 21 13:04:50.733216 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:50.733053    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="01adce0f06f0bcec39c55a7024c43faaf9f0e44d1e1bab9deceaea579997182e"
Jul 21 13:04:50.733216 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:04:50.733015    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"
Jul 21 13:05:01.483609 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.483528    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8" podUID="a898f824-e2f9-4f13-989d-3364ad568cad" containerName="main" containerID="cri-o://fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f" gracePeriod=45
Jul 21 13:05:01.735603 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.735412    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"]
Jul 21 13:05:01.737353 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.737334    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:05:01.737444 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.737392    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-52a2ba-mbxcm"]
Jul 21 13:05:01.759735 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.759693    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a898f824-e2f9-4f13-989d-3364ad568cad-tls-certs\") pod \"a898f824-e2f9-4f13-989d-3364ad568cad\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") "
Jul 21 13:05:01.759918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.759757    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-home\") pod \"a898f824-e2f9-4f13-989d-3364ad568cad\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") "
Jul 21 13:05:01.759918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.759814    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-tmp-dir\") pod \"a898f824-e2f9-4f13-989d-3364ad568cad\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") "
Jul 21 13:05:01.759918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.759844    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfmjw\" (UniqueName: \"kubernetes.io/projected/a898f824-e2f9-4f13-989d-3364ad568cad-kube-api-access-bfmjw\") pod \"a898f824-e2f9-4f13-989d-3364ad568cad\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") "
Jul 21 13:05:01.759918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.759872    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-dshm\") pod \"a898f824-e2f9-4f13-989d-3364ad568cad\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") "
Jul 21 13:05:01.760143 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.759924    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-model-cache\") pod \"a898f824-e2f9-4f13-989d-3364ad568cad\" (UID: \"a898f824-e2f9-4f13-989d-3364ad568cad\") "
Jul 21 13:05:01.760616 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.760511    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a898f824-e2f9-4f13-989d-3364ad568cad" (UID: "a898f824-e2f9-4f13-989d-3364ad568cad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:01.760750 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.760712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-model-cache" (OuterVolumeSpecName: "model-cache") pod "a898f824-e2f9-4f13-989d-3364ad568cad" (UID: "a898f824-e2f9-4f13-989d-3364ad568cad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:01.761017 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.760948    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-home" (OuterVolumeSpecName: "home") pod "a898f824-e2f9-4f13-989d-3364ad568cad" (UID: "a898f824-e2f9-4f13-989d-3364ad568cad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:01.762377 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.762343    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a898f824-e2f9-4f13-989d-3364ad568cad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a898f824-e2f9-4f13-989d-3364ad568cad" (UID: "a898f824-e2f9-4f13-989d-3364ad568cad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:05:01.762926 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.762896    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a898f824-e2f9-4f13-989d-3364ad568cad-kube-api-access-bfmjw" (OuterVolumeSpecName: "kube-api-access-bfmjw") pod "a898f824-e2f9-4f13-989d-3364ad568cad" (UID: "a898f824-e2f9-4f13-989d-3364ad568cad"). InnerVolumeSpecName "kube-api-access-bfmjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:05:01.763379 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.763360    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-dshm" (OuterVolumeSpecName: "dshm") pod "a898f824-e2f9-4f13-989d-3364ad568cad" (UID: "a898f824-e2f9-4f13-989d-3364ad568cad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:01.786444 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.786413    2577 generic.go:356] "Generic (PLEG): container finished" podID="a898f824-e2f9-4f13-989d-3364ad568cad" containerID="fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f" exitCode=0
Jul 21 13:05:01.786617 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.786463    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"
Jul 21 13:05:01.786617 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.786494    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8" event={"ID":"a898f824-e2f9-4f13-989d-3364ad568cad","Type":"ContainerDied","Data":"fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f"}
Jul 21 13:05:01.786617 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.786555    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8" event={"ID":"a898f824-e2f9-4f13-989d-3364ad568cad","Type":"ContainerDied","Data":"ca40bf69d5b3bb1591f614b8771bfd6ab6f210bb7c622f8c8815b211e03dcb9b"}
Jul 21 13:05:01.786617 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.786579    2577 scope.go:117] "RemoveContainer" containerID="fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f"
Jul 21 13:05:01.795356 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.795335    2577 scope.go:117] "RemoveContainer" containerID="fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f"
Jul 21 13:05:01.795612 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:05:01.795588    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f\": container with ID starting with fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f not found: ID does not exist" containerID="fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f"
Jul 21 13:05:01.795676 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.795619    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f"} err="failed to get container status \"fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f\": rpc error: code = NotFound desc = could not find container \"fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f\": container with ID starting with fb57a613ea266c174094796d5b1928238cc93c916ca9f0383244ae9db28c061f not found: ID does not exist"
Jul 21 13:05:01.815991 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.815947    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"]
Jul 21 13:05:01.821285 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.821255    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0bdb53/test-model-0bdb53-kserve-5f977d66f7-5bxk8"]
Jul 21 13:05:01.860662 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.860620    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-dshm\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:01.860890 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.860677    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-model-cache\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:01.860890 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.860690    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a898f824-e2f9-4f13-989d-3364ad568cad-tls-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:01.860890 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.860699    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-home\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:01.860890 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.860707    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a898f824-e2f9-4f13-989d-3364ad568cad-tmp-dir\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:01.860890 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:01.860716    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfmjw\" (UniqueName: \"kubernetes.io/projected/a898f824-e2f9-4f13-989d-3364ad568cad-kube-api-access-bfmjw\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:02.833697 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:02.833663    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="579d9289-7003-44c2-b208-b7455825a95a" path="/var/lib/kubelet/pods/579d9289-7003-44c2-b208-b7455825a95a/volumes"
Jul 21 13:05:02.834248 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:02.834233    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a898f824-e2f9-4f13-989d-3364ad568cad" path="/var/lib/kubelet/pods/a898f824-e2f9-4f13-989d-3364ad568cad/volumes"
Jul 21 13:05:08.075158 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:08.075069    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"]
Jul 21 13:05:08.075614 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:08.075275    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy" containerID="cri-o://bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a" gracePeriod=30
Jul 21 13:05:13.327643 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.327616    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:05:13.362760 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362674    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-data\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.362760 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362722    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-certs\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.362760 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362752    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-token\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.363086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362781    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-envoy\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.363086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362811    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-credential-socket\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.363086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-podinfo\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.363086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362868    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istiod-ca-cert\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.363086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362893    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z26nh\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-kube-api-access-z26nh\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.363086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.362942    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-socket\") pod \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\" (UID: \"9cc0cd7b-07ae-4621-a54a-abdc5070863d\") "
Jul 21 13:05:13.363086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.363039    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:13.363352 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.363265    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.363352 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.363330    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:05:13.363457 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.363435    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:13.363576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.363533    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-data" (OuterVolumeSpecName: "istio-data") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:13.363721 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.363589    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:13.365384 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.365355    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-token" (OuterVolumeSpecName: "istio-token") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:05:13.365507 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.365455    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:13.365594 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.365573    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:05:13.365637 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.365621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-kube-api-access-z26nh" (OuterVolumeSpecName: "kube-api-access-z26nh") pod "9cc0cd7b-07ae-4621-a54a-abdc5070863d" (UID: "9cc0cd7b-07ae-4621-a54a-abdc5070863d"). InnerVolumeSpecName "kube-api-access-z26nh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:05:13.464269 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464232    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.464269 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464264    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.464269 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464275    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.464516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464285    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.464516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464295    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.464516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464303    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9cc0cd7b-07ae-4621-a54a-abdc5070863d-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.464516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464312    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z26nh\" (UniqueName: \"kubernetes.io/projected/9cc0cd7b-07ae-4621-a54a-abdc5070863d-kube-api-access-z26nh\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.464516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.464322    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9cc0cd7b-07ae-4621-a54a-abdc5070863d-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:13.848264 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.848228    2577 generic.go:356] "Generic (PLEG): container finished" podID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerID="bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a" exitCode=0
Jul 21 13:05:13.848442 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.848280    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"
Jul 21 13:05:13.848442 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.848309    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" event={"ID":"9cc0cd7b-07ae-4621-a54a-abdc5070863d","Type":"ContainerDied","Data":"bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a"}
Jul 21 13:05:13.848442 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.848361    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh" event={"ID":"9cc0cd7b-07ae-4621-a54a-abdc5070863d","Type":"ContainerDied","Data":"59267ad460bd4e0db4f970d6f1db2d0db7cf982a905ce0e9f414bc1cf54baee3"}
Jul 21 13:05:13.848442 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.848383    2577 scope.go:117] "RemoveContainer" containerID="bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a"
Jul 21 13:05:13.857387 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.857370    2577 scope.go:117] "RemoveContainer" containerID="fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5"
Jul 21 13:05:13.865754 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.865733    2577 scope.go:117] "RemoveContainer" containerID="bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a"
Jul 21 13:05:13.866055 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:05:13.866030    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a\": container with ID starting with bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a not found: ID does not exist" containerID="bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a"
Jul 21 13:05:13.866128 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.866070    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a"} err="failed to get container status \"bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a\": rpc error: code = NotFound desc = could not find container \"bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a\": container with ID starting with bb73304badcb94cc94fd02e971657e44d2e18622090c282f38b29a26ec83de8a not found: ID does not exist"
Jul 21 13:05:13.866128 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.866096    2577 scope.go:117] "RemoveContainer" containerID="fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5"
Jul 21 13:05:13.866371 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:05:13.866350    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5\": container with ID starting with fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5 not found: ID does not exist" containerID="fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5"
Jul 21 13:05:13.866420 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.866378    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5"} err="failed to get container status \"fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5\": rpc error: code = NotFound desc = could not find container \"fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5\": container with ID starting with fc553bf547283a2a82a01c802658b248c8686d91c29eca6f28b977e59e00c6b5 not found: ID does not exist"
Jul 21 13:05:13.870138 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.870116    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"]
Jul 21 13:05:13.874199 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:13.874179    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-52a2ba-openshift-default-669dbbc959-5mmmh"]
Jul 21 13:05:14.833310 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:14.833273    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" path="/var/lib/kubelet/pods/9cc0cd7b-07ae-4621-a54a-abdc5070863d/volumes"
Jul 21 13:05:30.365762 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:30.365716    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"]
Jul 21 13:05:30.366364 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:30.366062    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy" containerID="cri-o://be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee" gracePeriod=30
Jul 21 13:05:31.314661 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.314630    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"]
Jul 21 13:05:31.315005 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.314993    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="579d9289-7003-44c2-b208-b7455825a95a" containerName="revoke-keys"
Jul 21 13:05:31.315064 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315007    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="579d9289-7003-44c2-b208-b7455825a95a" containerName="revoke-keys"
Jul 21 13:05:31.315064 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315032    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy"
Jul 21 13:05:31.315064 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315040    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy"
Jul 21 13:05:31.315064 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315054    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a898f824-e2f9-4f13-989d-3364ad568cad" containerName="main"
Jul 21 13:05:31.315064 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315060    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a898f824-e2f9-4f13-989d-3364ad568cad" containerName="main"
Jul 21 13:05:31.315225 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315069    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy"
Jul 21 13:05:31.315225 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315075    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy"
Jul 21 13:05:31.315225 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315127    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy"
Jul 21 13:05:31.315225 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315136    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="579d9289-7003-44c2-b208-b7455825a95a" containerName="revoke-keys"
Jul 21 13:05:31.315225 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.315142    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a898f824-e2f9-4f13-989d-3364ad568cad" containerName="main"
Jul 21 13:05:31.317696 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.317680    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.320573 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.320547    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-019b5c-openshift-default-dockercfg-8pmqq\""
Jul 21 13:05:31.332153 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.332125    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"]
Jul 21 13:05:31.439395 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-token\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439395 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439397    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-envoy\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439851 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439428    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istiod-ca-cert\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439851 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439452    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-credential-socket\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439851 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phwzg\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-kube-api-access-phwzg\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439851 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-data\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439851 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439637    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-certs\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439851 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-socket\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.439851 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.439771    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-podinfo\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540547 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540499    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-envoy\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540547 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540549    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istiod-ca-cert\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540575    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-credential-socket\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540598    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phwzg\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-kube-api-access-phwzg\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-data\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540656    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-certs\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540686    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-socket\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-podinfo\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.540831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.540818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-token\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.541229 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.541121    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-data\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.541229 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.541138    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-credential-socket\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.541315 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.541221    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-socket\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.541398 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.541375    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-certs\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.541757 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.541738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istiod-ca-cert\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.543299 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.543273    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-envoy\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.543411 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.543335    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-podinfo\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.549610 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.549581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-token\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.549826 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.549805    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phwzg\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-kube-api-access-phwzg\") pod \"e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") " pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.629093 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.628986    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:31.776920 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.776890    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"]
Jul 21 13:05:31.779612 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:05:31.779520    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb6825a_410a_43bd_9ce8_bd11d027a1a8.slice/crio-258d6a0d1ff480cc05462a648705bef4c6c1099a5f27ea3a4601f557491b0cf1 WatchSource:0}: Error finding container 258d6a0d1ff480cc05462a648705bef4c6c1099a5f27ea3a4601f557491b0cf1: Status 404 returned error can't find the container with id 258d6a0d1ff480cc05462a648705bef4c6c1099a5f27ea3a4601f557491b0cf1
Jul 21 13:05:31.782286 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.782252    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:31.782389 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.782322    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:31.782389 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.782351    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:31.938248 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.938158    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" event={"ID":"2cb6825a-410a-43bd-9ce8-bd11d027a1a8","Type":"ContainerStarted","Data":"ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90"}
Jul 21 13:05:31.938398 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.938263    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" event={"ID":"2cb6825a-410a-43bd-9ce8-bd11d027a1a8","Type":"ContainerStarted","Data":"258d6a0d1ff480cc05462a648705bef4c6c1099a5f27ea3a4601f557491b0cf1"}
Jul 21 13:05:31.960269 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:31.960203    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podStartSLOduration=0.960180788 podStartE2EDuration="960.180788ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:05:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:05:31.958905251 +0000 UTC m=+2269.743920239" watchObservedRunningTime="2026-07-21 13:05:31.960180788 +0000 UTC m=+2269.745195755"
Jul 21 13:05:32.629483 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:32.629439    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:32.634594 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:32.634567    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:32.943338 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:32.943254    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:32.944293 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:32.944274    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:35.616885 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.616860    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:05:35.778144 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778115    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-data\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778343 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778163    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-podinfo\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778343 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778180    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-token\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778343 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778222    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-socket\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778343 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778262    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-credential-socket\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778343 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778305    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-envoy\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778343 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778332    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cmk5s\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-kube-api-access-cmk5s\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778673 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778385    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istiod-ca-cert\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778673 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778448    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-certs\") pod \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\" (UID: \"8705b482-3677-4556-b25f-8ab6ce3d8bb1\") "
Jul 21 13:05:35.778673 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778604    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:35.778860 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778733    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.778860 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778827    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:35.778860 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.778845    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:35.779053 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.779022    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-data" (OuterVolumeSpecName: "istio-data") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:35.779116 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.779073    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:05:35.780500 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.780475    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-kube-api-access-cmk5s" (OuterVolumeSpecName: "kube-api-access-cmk5s") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "kube-api-access-cmk5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:05:35.780633 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.780586    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:05:35.780865 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.780846    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:05:35.780910 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.780882    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8705b482-3677-4556-b25f-8ab6ce3d8bb1" (UID: "8705b482-3677-4556-b25f-8ab6ce3d8bb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:05:35.879852 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879808    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.879852 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879845    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cmk5s\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-kube-api-access-cmk5s\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.879852 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879855    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.879852 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879863    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.880160 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879873    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.880160 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879880    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.880160 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879888    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8705b482-3677-4556-b25f-8ab6ce3d8bb1-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.880160 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.879896    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8705b482-3677-4556-b25f-8ab6ce3d8bb1-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:05:35.959090 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.959055    2577 generic.go:356] "Generic (PLEG): container finished" podID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerID="be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee" exitCode=0
Jul 21 13:05:35.959270 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.959106    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"
Jul 21 13:05:35.959270 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.959131    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" event={"ID":"8705b482-3677-4556-b25f-8ab6ce3d8bb1","Type":"ContainerDied","Data":"be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee"}
Jul 21 13:05:35.959270 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.959182    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv" event={"ID":"8705b482-3677-4556-b25f-8ab6ce3d8bb1","Type":"ContainerDied","Data":"7ca044a539068b35876e5f8eb934e42c700afa34459a52f77a265f57d4b2aa2c"}
Jul 21 13:05:35.959270 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.959206    2577 scope.go:117] "RemoveContainer" containerID="be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee"
Jul 21 13:05:35.968310 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.968289    2577 scope.go:117] "RemoveContainer" containerID="bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197"
Jul 21 13:05:35.976711 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.976691    2577 scope.go:117] "RemoveContainer" containerID="be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee"
Jul 21 13:05:35.977001 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:05:35.976981    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee\": container with ID starting with be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee not found: ID does not exist" containerID="be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee"
Jul 21 13:05:35.977058 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.977012    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee"} err="failed to get container status \"be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee\": rpc error: code = NotFound desc = could not find container \"be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee\": container with ID starting with be86668c71ded38aa7a81b15b2a0bb6fe1552c8665d0b9e3d4231d00d9014dee not found: ID does not exist"
Jul 21 13:05:35.977058 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.977031    2577 scope.go:117] "RemoveContainer" containerID="bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197"
Jul 21 13:05:35.978068 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:05:35.977735    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197\": container with ID starting with bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197 not found: ID does not exist" containerID="bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197"
Jul 21 13:05:35.978068 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.977852    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197"} err="failed to get container status \"bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197\": rpc error: code = NotFound desc = could not find container \"bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197\": container with ID starting with bf33bcf605c320a6232d8756eab38d1112f1294eae01d714c49ef00970d1c197 not found: ID does not exist"
Jul 21 13:05:35.983788 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.983753    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"]
Jul 21 13:05:35.988079 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:35.988053    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0bdb53-openshift-default-7d4c964fc5-qsddv"]
Jul 21 13:05:36.833662 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:36.833624    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" path="/var/lib/kubelet/pods/8705b482-3677-4556-b25f-8ab6ce3d8bb1/volumes"
Jul 21 13:05:39.339468 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.339429    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"]
Jul 21 13:05:39.339863 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.339819    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy"
Jul 21 13:05:39.339863 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.339830    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy"
Jul 21 13:05:39.339966 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.339897    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cc0cd7b-07ae-4621-a54a-abdc5070863d" containerName="istio-proxy"
Jul 21 13:05:39.339966 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.339906    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy"
Jul 21 13:05:39.339966 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.339911    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy"
Jul 21 13:05:39.343044 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.343006    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.345433 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.345405    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-69d4e8-openshift-default-dockercfg-6mjxk\""
Jul 21 13:05:39.355906 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.355867    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"]
Jul 21 13:05:39.514159 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514121    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-data\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514344 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514187    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589b30b6-e36e-4acd-941b-18a93f219a72-istio-podinfo\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514344 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589b30b6-e36e-4acd-941b-18a93f219a72-istiod-ca-cert\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514344 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxlwv\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-kube-api-access-vxlwv\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514344 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514284    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-credential-socket\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514344 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-certs\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514598 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514416    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-socket\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514598 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-istio-token\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.514598 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.514494    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-envoy\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615526 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-data\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615526 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589b30b6-e36e-4acd-941b-18a93f219a72-istio-podinfo\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615526 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589b30b6-e36e-4acd-941b-18a93f219a72-istiod-ca-cert\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615526 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxlwv\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-kube-api-access-vxlwv\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615870 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-credential-socket\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615870 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615581    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-certs\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615870 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-socket\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615870 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615641    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-istio-token\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.615870 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615699    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-envoy\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.616185 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615895    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-data\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.616185 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.615940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-credential-socket\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.616185 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.616041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-certs\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.616185 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.616172    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-socket\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.616388 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.616310    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589b30b6-e36e-4acd-941b-18a93f219a72-istiod-ca-cert\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.617941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.617908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-envoy\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.618060 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.618046    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589b30b6-e36e-4acd-941b-18a93f219a72-istio-podinfo\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.624601 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.624580    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-istio-token\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.625089 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.625062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxlwv\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-kube-api-access-vxlwv\") pod \"e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") " pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.656035 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.656002    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:39.794555 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.794531    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"]
Jul 21 13:05:39.796563 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:05:39.796536    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod589b30b6_e36e_4acd_941b_18a93f219a72.slice/crio-0cb89d56cdfbdb8437309b24fd20bc30406b1b62f7341c0dc2e897278db15719 WatchSource:0}: Error finding container 0cb89d56cdfbdb8437309b24fd20bc30406b1b62f7341c0dc2e897278db15719: Status 404 returned error can't find the container with id 0cb89d56cdfbdb8437309b24fd20bc30406b1b62f7341c0dc2e897278db15719
Jul 21 13:05:39.798526 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.798487    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:39.798629 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.798553    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:39.798629 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.798596    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:39.984015 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.983980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" event={"ID":"589b30b6-e36e-4acd-941b-18a93f219a72","Type":"ContainerStarted","Data":"a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68"}
Jul 21 13:05:39.984189 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:39.984031    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" event={"ID":"589b30b6-e36e-4acd-941b-18a93f219a72","Type":"ContainerStarted","Data":"0cb89d56cdfbdb8437309b24fd20bc30406b1b62f7341c0dc2e897278db15719"}
Jul 21 13:05:40.009296 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:40.009244    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" podStartSLOduration=1.009227202 podStartE2EDuration="1.009227202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:05:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:05:40.007689542 +0000 UTC m=+2277.792704537" watchObservedRunningTime="2026-07-21 13:05:40.009227202 +0000 UTC m=+2277.794242168"
Jul 21 13:05:40.656874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:40.656830    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:40.662394 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:40.662364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:40.989479 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:40.989451    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:40.990389 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:40.990369    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:05:48.985163 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:48.985120    2577 patch_prober.go:28] interesting pod/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 21 13:05:48.985628 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:48.985196    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 21 13:05:50.037297 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:50.037260    2577 generic.go:356] "Generic (PLEG): container finished" podID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerID="ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90" exitCode=0
Jul 21 13:05:50.037825 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:50.037334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" event={"ID":"2cb6825a-410a-43bd-9ce8-bd11d027a1a8","Type":"ContainerDied","Data":"ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90"}
Jul 21 13:05:50.037898 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:50.037838    2577 scope.go:117] "RemoveContainer" containerID="ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90"
Jul 21 13:05:50.039114 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:50.039081    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:50.039214 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:50.039153    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:50.039214 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:50.039188    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:51.046506 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:51.046472    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" event={"ID":"2cb6825a-410a-43bd-9ce8-bd11d027a1a8","Type":"ContainerStarted","Data":"567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5"}
Jul 21 13:05:51.629702 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:51.629603    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:51.629929 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:51.629892    2577 patch_prober.go:28] interesting pod/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 13:05:51.630029 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:51.629947    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 13:05:52.630477 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:52.630436    2577 patch_prober.go:28] interesting pod/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 13:05:52.631135 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:52.631099    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 13:05:53.629946 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:53.629903    2577 patch_prober.go:28] interesting pod/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 13:05:53.630274 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:53.630237    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 13:05:54.630202 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:54.630159    2577 patch_prober.go:28] interesting pod/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 21 13:05:54.630818 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:54.630785    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 21 13:05:55.639927 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:55.638504    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:55.639927 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:55.639772    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:55.644576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:55.644552    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:05:59.094330 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:59.094292    2577 generic.go:356] "Generic (PLEG): container finished" podID="589b30b6-e36e-4acd-941b-18a93f219a72" containerID="a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68" exitCode=0
Jul 21 13:05:59.094749 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:59.094365    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" event={"ID":"589b30b6-e36e-4acd-941b-18a93f219a72","Type":"ContainerDied","Data":"a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68"}
Jul 21 13:05:59.094792 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:59.094768    2577 scope.go:117] "RemoveContainer" containerID="a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68"
Jul 21 13:05:59.095802 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:59.095771    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:59.095874 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:59.095847    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:05:59.095915 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:05:59.095878    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:06:00.101616 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:00.101580    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" event={"ID":"589b30b6-e36e-4acd-941b-18a93f219a72","Type":"ContainerStarted","Data":"79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2"}
Jul 21 13:06:00.656233 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:00.656196    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:06:00.656496 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:00.656470    2577 patch_prober.go:28] interesting pod/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 21 13:06:00.656540 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:00.656521    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 21 13:06:01.656708 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:01.656674    2577 patch_prober.go:28] interesting pod/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 21 13:06:01.657124 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:01.656729    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 21 13:06:02.656503 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:02.656464    2577 patch_prober.go:28] interesting pod/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 21 13:06:02.656699 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:02.656520    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 21 13:06:03.657188 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:03.657144    2577 patch_prober.go:28] interesting pod/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 21 13:06:03.657679 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:03.657253    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 21 13:06:04.660411 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:04.660377    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:06:04.660895 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:04.660794    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:06:04.661538 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:04.661521    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:06:21.469935 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:21.469896    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"]
Jul 21 13:06:21.470488 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:21.470222    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy" containerID="cri-o://567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5" gracePeriod=30
Jul 21 13:06:25.641721 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:25.641677    2577 patch_prober.go:28] interesting pod/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 13:06:25.642161 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:25.641773    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 13:06:26.723661 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.723632    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:06:26.760735 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760702    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-credential-socket\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.760735 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760736    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-data\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.760993 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760761    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-envoy\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.760993 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760801    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phwzg\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-kube-api-access-phwzg\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.760993 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760827    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-token\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.760993 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760859    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-podinfo\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.760993 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760891    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istiod-ca-cert\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.760993 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.760946    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-certs\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.761327 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761007    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-socket\") pod \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\" (UID: \"2cb6825a-410a-43bd-9ce8-bd11d027a1a8\") "
Jul 21 13:06:26.761327 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761025    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:26.761327 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:26.761327 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761296    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.761327 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761313    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.761586 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761452    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:26.761662 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761630    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:06:26.761802 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.761679    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-data" (OuterVolumeSpecName: "istio-data") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:26.763300 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.763269    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:06:26.763412 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.763326    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-token" (OuterVolumeSpecName: "istio-token") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:06:26.763412 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.763389    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-kube-api-access-phwzg" (OuterVolumeSpecName: "kube-api-access-phwzg") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "kube-api-access-phwzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:06:26.763611 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.763589    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2cb6825a-410a-43bd-9ce8-bd11d027a1a8" (UID: "2cb6825a-410a-43bd-9ce8-bd11d027a1a8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:26.862516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.862486    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.862516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.862512    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.862516 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.862523    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phwzg\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-kube-api-access-phwzg\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.862749 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.862533    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.862749 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.862542    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.862749 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.862550    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:26.862749 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:26.862558    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cb6825a-410a-43bd-9ce8-bd11d027a1a8-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:27.236925 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.236890    2577 generic.go:356] "Generic (PLEG): container finished" podID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerID="567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5" exitCode=0
Jul 21 13:06:27.237137 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.236984    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"
Jul 21 13:06:27.237137 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.236994    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" event={"ID":"2cb6825a-410a-43bd-9ce8-bd11d027a1a8","Type":"ContainerDied","Data":"567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5"}
Jul 21 13:06:27.237137 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.237053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb" event={"ID":"2cb6825a-410a-43bd-9ce8-bd11d027a1a8","Type":"ContainerDied","Data":"258d6a0d1ff480cc05462a648705bef4c6c1099a5f27ea3a4601f557491b0cf1"}
Jul 21 13:06:27.237137 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.237076    2577 scope.go:117] "RemoveContainer" containerID="567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5"
Jul 21 13:06:27.245857 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.245839    2577 scope.go:117] "RemoveContainer" containerID="ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90"
Jul 21 13:06:27.253796 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.253769    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"]
Jul 21 13:06:27.254114 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.254100    2577 scope.go:117] "RemoveContainer" containerID="567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5"
Jul 21 13:06:27.254393 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:06:27.254374    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5\": container with ID starting with 567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5 not found: ID does not exist" containerID="567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5"
Jul 21 13:06:27.254441 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.254403    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5"} err="failed to get container status \"567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5\": rpc error: code = NotFound desc = could not find container \"567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5\": container with ID starting with 567d14107f52a4572cb424e2115cd53c455953003d74b37a413ab07f22f10ba5 not found: ID does not exist"
Jul 21 13:06:27.254441 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.254422    2577 scope.go:117] "RemoveContainer" containerID="ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90"
Jul 21 13:06:27.254664 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:06:27.254645    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90\": container with ID starting with ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90 not found: ID does not exist" containerID="ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90"
Jul 21 13:06:27.254711 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.254673    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90"} err="failed to get container status \"ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90\": rpc error: code = NotFound desc = could not find container \"ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90\": container with ID starting with ac93dc0522256acb41c83f1c3420acbdc88fcb8027fe188690a853220e45cb90 not found: ID does not exist"
Jul 21 13:06:27.258353 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:27.258332    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-019b5c-openshift-default-6c64bd79bc-cz8pb"]
Jul 21 13:06:28.832209 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:28.832172    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" path="/var/lib/kubelet/pods/2cb6825a-410a-43bd-9ce8-bd11d027a1a8/volumes"
Jul 21 13:06:48.449902 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:48.449861    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"]
Jul 21 13:06:48.450379 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:48.450185    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy" containerID="cri-o://79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2" gracePeriod=30
Jul 21 13:06:49.347602 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.347564    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"]
Jul 21 13:06:49.348137 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348118    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy"
Jul 21 13:06:49.348137 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348137    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8705b482-3677-4556-b25f-8ab6ce3d8bb1" containerName="istio-proxy"
Jul 21 13:06:49.348307 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348155    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy"
Jul 21 13:06:49.348307 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348161    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy"
Jul 21 13:06:49.348307 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348173    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy"
Jul 21 13:06:49.348307 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348178    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy"
Jul 21 13:06:49.348307 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348234    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy"
Jul 21 13:06:49.348307 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.348244    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb6825a-410a-43bd-9ce8-bd11d027a1a8" containerName="istio-proxy"
Jul 21 13:06:49.350288 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.350269    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.352575 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.352555    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-3cf59c-openshift-default-dockercfg-k5r4x\""
Jul 21 13:06:49.366032 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.366001    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"]
Jul 21 13:06:49.459329 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.459290    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istiod-ca-cert\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.459920 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.459876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-token\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.460098 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.459940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-podinfo\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.460098 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.460009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-certs\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.460098 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.460042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-socket\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.460098 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.460063    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-envoy\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.460394 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.460116    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-credential-socket\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.460394 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.460169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8hc52\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-kube-api-access-8hc52\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.460394 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.460217    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-data\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561418 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-data\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561418 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561425    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istiod-ca-cert\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561692 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561449    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-token\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561692 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561473    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-podinfo\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561692 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561509    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-certs\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561692 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561547    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-socket\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561692 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-envoy\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561692 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-credential-socket\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.561692 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561631    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8hc52\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-kube-api-access-8hc52\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.562074 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-data\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.562074 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561926    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-certs\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.562074 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.561936    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-socket\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.562208 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.562188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-credential-socket\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.562528 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.562504    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istiod-ca-cert\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.563917 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.563900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-envoy\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.564404 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.564387    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-podinfo\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.570114 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.570077    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-token\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.570239 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.570171    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8hc52\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-kube-api-access-8hc52\") pod \"e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") " pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.661187 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.661087    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:49.663174 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.663141    2577 patch_prober.go:28] interesting pod/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 21 13:06:49.663317 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.663207    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 21 13:06:49.808346 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.808317    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"]
Jul 21 13:06:49.810744 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:06:49.810715    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb66cccae_c3e7_4e6f_a150_200ad5c5b8e7.slice/crio-5226056befaecb636ca3588c3a8daf4124f235cf13a133750b01be9d210782d6 WatchSource:0}: Error finding container 5226056befaecb636ca3588c3a8daf4124f235cf13a133750b01be9d210782d6: Status 404 returned error can't find the container with id 5226056befaecb636ca3588c3a8daf4124f235cf13a133750b01be9d210782d6
Jul 21 13:06:49.812934 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.812902    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:06:49.813031 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.812999    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:06:49.813073 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:49.813033    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:06:50.348523 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:50.348466    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" event={"ID":"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7","Type":"ContainerStarted","Data":"792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1"}
Jul 21 13:06:50.348736 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:50.348554    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" event={"ID":"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7","Type":"ContainerStarted","Data":"5226056befaecb636ca3588c3a8daf4124f235cf13a133750b01be9d210782d6"}
Jul 21 13:06:50.370766 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:50.370710    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" podStartSLOduration=1.3706911449999999 podStartE2EDuration="1.370691145s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:06:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:06:50.368446994 +0000 UTC m=+2348.153461981" watchObservedRunningTime="2026-07-21 13:06:50.370691145 +0000 UTC m=+2348.155706112"
Jul 21 13:06:50.661586 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:50.661490    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:50.666432 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:50.666405    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:51.353379 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:51.353349    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:51.354430 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:51.354412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:06:53.706206 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.706180    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:06:53.803201 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-data\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803201 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803201    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-certs\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803455 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803241    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589b30b6-e36e-4acd-941b-18a93f219a72-istio-podinfo\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803455 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803269    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-envoy\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803455 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803311    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-credential-socket\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803455 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803337    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-istio-token\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803455 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803381    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-socket\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803706 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803469    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vxlwv\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-kube-api-access-vxlwv\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803706 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803509    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589b30b6-e36e-4acd-941b-18a93f219a72-istiod-ca-cert\") pod \"589b30b6-e36e-4acd-941b-18a93f219a72\" (UID: \"589b30b6-e36e-4acd-941b-18a93f219a72\") "
Jul 21 13:06:53.803706 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803603    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:53.803870 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803712    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:53.803929 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803875    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.803929 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803883    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/589b30b6-e36e-4acd-941b-18a93f219a72-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:06:53.803929 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.803897    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.804133 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.804080    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:53.804244 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.804224    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-data" (OuterVolumeSpecName: "istio-data") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:53.805785 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.805696    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-istio-token" (OuterVolumeSpecName: "istio-token") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:06:53.805785 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.805762    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:06:53.806060 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.806042    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-kube-api-access-vxlwv" (OuterVolumeSpecName: "kube-api-access-vxlwv") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "kube-api-access-vxlwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:06:53.806135 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.806072    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/589b30b6-e36e-4acd-941b-18a93f219a72-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "589b30b6-e36e-4acd-941b-18a93f219a72" (UID: "589b30b6-e36e-4acd-941b-18a93f219a72"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:06:53.905277 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.905239    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vxlwv\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-kube-api-access-vxlwv\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.905277 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.905272    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/589b30b6-e36e-4acd-941b-18a93f219a72-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.905277 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.905282    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.905523 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.905293    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/589b30b6-e36e-4acd-941b-18a93f219a72-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.905523 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.905303    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.905523 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.905313    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/589b30b6-e36e-4acd-941b-18a93f219a72-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:53.905523 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:53.905321    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/589b30b6-e36e-4acd-941b-18a93f219a72-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:06:54.370090 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.370056    2577 generic.go:356] "Generic (PLEG): container finished" podID="589b30b6-e36e-4acd-941b-18a93f219a72" containerID="79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2" exitCode=0
Jul 21 13:06:54.370277 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.370104    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"
Jul 21 13:06:54.370277 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.370135    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" event={"ID":"589b30b6-e36e-4acd-941b-18a93f219a72","Type":"ContainerDied","Data":"79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2"}
Jul 21 13:06:54.370277 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.370187    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9" event={"ID":"589b30b6-e36e-4acd-941b-18a93f219a72","Type":"ContainerDied","Data":"0cb89d56cdfbdb8437309b24fd20bc30406b1b62f7341c0dc2e897278db15719"}
Jul 21 13:06:54.370277 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.370209    2577 scope.go:117] "RemoveContainer" containerID="79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2"
Jul 21 13:06:54.379204 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.379186    2577 scope.go:117] "RemoveContainer" containerID="a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68"
Jul 21 13:06:54.387087 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.387066    2577 scope.go:117] "RemoveContainer" containerID="79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2"
Jul 21 13:06:54.387375 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:06:54.387351    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2\": container with ID starting with 79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2 not found: ID does not exist" containerID="79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2"
Jul 21 13:06:54.387479 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.387387    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2"} err="failed to get container status \"79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2\": rpc error: code = NotFound desc = could not find container \"79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2\": container with ID starting with 79afb86e95f53fadad27a5775be25eec951e8e9299bf4aaa16a1514c976ae5b2 not found: ID does not exist"
Jul 21 13:06:54.387479 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.387410    2577 scope.go:117] "RemoveContainer" containerID="a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68"
Jul 21 13:06:54.387671 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:06:54.387650    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68\": container with ID starting with a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68 not found: ID does not exist" containerID="a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68"
Jul 21 13:06:54.387708 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.387678    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68"} err="failed to get container status \"a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68\": rpc error: code = NotFound desc = could not find container \"a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68\": container with ID starting with a55a9396c5285f4538468c712f09bd6144bb9929a7272f1ac9506dcd80ca9b68 not found: ID does not exist"
Jul 21 13:06:54.394999 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.394977    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"]
Jul 21 13:06:54.398942 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.398922    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-69d4e8-openshift-default-688695fdbf-2k5j9"]
Jul 21 13:06:54.834622 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:54.834587    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" path="/var/lib/kubelet/pods/589b30b6-e36e-4acd-941b-18a93f219a72/volumes"
Jul 21 13:06:57.400765 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.400732    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"]
Jul 21 13:06:57.401258 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.401088    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy"
Jul 21 13:06:57.401258 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.401099    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy"
Jul 21 13:06:57.401258 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.401117    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy"
Jul 21 13:06:57.401258 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.401123    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy"
Jul 21 13:06:57.401258 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.401176    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy"
Jul 21 13:06:57.401258 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.401187    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="589b30b6-e36e-4acd-941b-18a93f219a72" containerName="istio-proxy"
Jul 21 13:06:57.405698 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.405678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.408054 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.408034    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-b1b184-openshift-default-dockercfg-5bfl4\""
Jul 21 13:06:57.418086 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.418063    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"]
Jul 21 13:06:57.539246 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-socket\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539447 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-token\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539447 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539362    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-podinfo\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539447 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539402    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-credential-socket\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539447 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539435    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998141f8-a8cf-4f3c-82f0-5888fa87a053-istiod-ca-cert\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539619 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-certs\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539619 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-envoy\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539619 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-data\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.539619 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.539590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqgnx\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-kube-api-access-sqgnx\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.640827 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.640783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-socket\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.640827 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.640827    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-token\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641115 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.640863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-podinfo\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641115 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.640891    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-credential-socket\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641115 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.640918    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998141f8-a8cf-4f3c-82f0-5888fa87a053-istiod-ca-cert\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641115 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.640948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-certs\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641115 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641028    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-envoy\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641115 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-data\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641424 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641082    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqgnx\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-kube-api-access-sqgnx\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641424 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641376    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-credential-socket\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641424 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641322    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-socket\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641557    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-data\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641640 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-certs\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.641783 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.641759    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998141f8-a8cf-4f3c-82f0-5888fa87a053-istiod-ca-cert\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.643315 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.643289    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-envoy\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.643553 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.643531    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-podinfo\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.648838 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.648816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-token\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.648917 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.648889    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqgnx\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-kube-api-access-sqgnx\") pod \"e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") " pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.717648 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.717561    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:57.868784 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.866216    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"]
Jul 21 13:06:57.869541 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:06:57.869513    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod998141f8_a8cf_4f3c_82f0_5888fa87a053.slice/crio-2d8b7715bd7feedb44b2a64f4bf013bdb85827da0ad561a59ab9385937ccb60f WatchSource:0}: Error finding container 2d8b7715bd7feedb44b2a64f4bf013bdb85827da0ad561a59ab9385937ccb60f: Status 404 returned error can't find the container with id 2d8b7715bd7feedb44b2a64f4bf013bdb85827da0ad561a59ab9385937ccb60f
Jul 21 13:06:57.871915 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.871876    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:06:57.872013 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.871994    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:06:57.872060 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:57.872043    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:06:58.392208 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:58.392167    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" event={"ID":"998141f8-a8cf-4f3c-82f0-5888fa87a053","Type":"ContainerStarted","Data":"95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f"}
Jul 21 13:06:58.392388 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:58.392229    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" event={"ID":"998141f8-a8cf-4f3c-82f0-5888fa87a053","Type":"ContainerStarted","Data":"2d8b7715bd7feedb44b2a64f4bf013bdb85827da0ad561a59ab9385937ccb60f"}
Jul 21 13:06:58.413519 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:58.413471    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" podStartSLOduration=1.413454705 podStartE2EDuration="1.413454705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:06:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:06:58.412214944 +0000 UTC m=+2356.197229909" watchObservedRunningTime="2026-07-21 13:06:58.413454705 +0000 UTC m=+2356.198469672"
Jul 21 13:06:58.718514 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:58.718419    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:58.723904 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:58.723875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:59.397116 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:59.397086    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:06:59.397996 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:06:59.397977    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:07:07.395174 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:07.395126    2577 patch_prober.go:28] interesting pod/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 21 13:07:07.395581 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:07.395203    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 21 13:07:08.443083 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:08.442979    2577 generic.go:356] "Generic (PLEG): container finished" podID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerID="792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1" exitCode=0
Jul 21 13:07:08.443083 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:08.443040    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" event={"ID":"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7","Type":"ContainerDied","Data":"792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1"}
Jul 21 13:07:08.443523 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:08.443440    2577 scope.go:117] "RemoveContainer" containerID="792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1"
Jul 21 13:07:08.444447 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:08.444415    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:07:08.444546 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:08.444502    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:07:08.444610 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:08.444548    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:07:09.452441 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:09.452407    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" event={"ID":"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7","Type":"ContainerStarted","Data":"0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d"}
Jul 21 13:07:09.661738 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:09.661701    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:07:09.662121 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:09.662096    2577 patch_prober.go:28] interesting pod/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:07:09.662309 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:09.662285    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 13:07:10.662517 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:10.662470    2577 patch_prober.go:28] interesting pod/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:07:10.663042 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:10.662539    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 13:07:11.661567 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:11.661525    2577 patch_prober.go:28] interesting pod/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:07:11.661864 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:11.661831    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 13:07:12.662518 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:12.662484    2577 patch_prober.go:28] interesting pod/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 21 13:07:12.663154 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:12.663118    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 21 13:07:13.665619 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:13.665591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:07:13.666052 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:13.665987    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:07:13.666730 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:13.666710    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:07:15.439127 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:15.439083    2577 patch_prober.go:28] interesting pod/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 21 13:07:15.439638 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:15.439155    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 21 13:07:17.503308 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:17.503275    2577 generic.go:356] "Generic (PLEG): container finished" podID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerID="95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f" exitCode=0
Jul 21 13:07:17.503850 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:17.503353    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" event={"ID":"998141f8-a8cf-4f3c-82f0-5888fa87a053","Type":"ContainerDied","Data":"95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f"}
Jul 21 13:07:17.503926 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:17.503856    2577 scope.go:117] "RemoveContainer" containerID="95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f"
Jul 21 13:07:17.504888 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:17.504862    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:07:17.505003 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:17.504925    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:07:17.505003 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:17.504976    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 21 13:07:18.509354 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:18.509322    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" event={"ID":"998141f8-a8cf-4f3c-82f0-5888fa87a053","Type":"ContainerStarted","Data":"b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262"}
Jul 21 13:07:18.718685 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:18.718648    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:07:18.718849 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:18.718836    2577 patch_prober.go:28] interesting pod/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 21 13:07:18.718898 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:18.718878    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 21 13:07:19.718480 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:19.718442    2577 patch_prober.go:28] interesting pod/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 21 13:07:19.718866 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:19.718499    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 21 13:07:20.717936 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:20.717897    2577 patch_prober.go:28] interesting pod/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 21 13:07:20.718168 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:20.717967    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 21 13:07:21.718243 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:21.718204    2577 patch_prober.go:28] interesting pod/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 21 13:07:21.718649 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:21.718265    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 21 13:07:22.721570 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:22.721536    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:07:22.721932 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:22.721871    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:07:22.722525 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:07:22.722508    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:08:01.501128 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.501093    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"]
Jul 21 13:08:01.503519 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.503497    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.508183 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.508155    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"kube-root-ca.crt\""
Jul 21 13:08:01.508183 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.508174    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"openshift-service-ca.crt\""
Jul 21 13:08:01.508365 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.508201    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"default-dockercfg-4mxf7\""
Jul 21 13:08:01.508365 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.508155    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"auth-test-model-b1b184-kserve-self-signed-certs\""
Jul 21 13:08:01.520089 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.520069    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"]
Jul 21 13:08:01.633185 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.633144    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-home\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.633386 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.633196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-dshm\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.633386 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.633224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tls-certs\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.633386 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.633276    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq8qz\" (UniqueName: \"kubernetes.io/projected/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-kube-api-access-bq8qz\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.633386 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.633312    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tmp-dir\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.633386 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.633337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-model-cache\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.734532 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.734496    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-home\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.734727 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.734544    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-dshm\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.734727 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.734564    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tls-certs\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.734727 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.734591    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq8qz\" (UniqueName: \"kubernetes.io/projected/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-kube-api-access-bq8qz\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.734727 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.734669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tmp-dir\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.734727 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.734715    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-model-cache\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.735058 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.734949    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-home\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.735174 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.735153    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-model-cache\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.735239 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.735177    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tmp-dir\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.736927 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.736901    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-dshm\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.737263 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.737245    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tls-certs\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.744084 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.744061    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq8qz\" (UniqueName: \"kubernetes.io/projected/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-kube-api-access-bq8qz\") pod \"auth-test-model-b1b184-kserve-7cd997899f-7zhvv\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") " pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.813219 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.813131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:01.952437 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:01.952410    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"]
Jul 21 13:08:01.954273 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:08:01.954243    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fa7d3e0_f4f7_4ff5_ac9d_4a7cce57e644.slice/crio-819069d85ef064e579e5af755c633dd476b58ffdf64c5bd5572fbeff43899acb WatchSource:0}: Error finding container 819069d85ef064e579e5af755c633dd476b58ffdf64c5bd5572fbeff43899acb: Status 404 returned error can't find the container with id 819069d85ef064e579e5af755c633dd476b58ffdf64c5bd5572fbeff43899acb
Jul 21 13:08:02.714824 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:02.714789    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv" event={"ID":"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644","Type":"ContainerStarted","Data":"30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e"}
Jul 21 13:08:02.714824 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:02.714829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv" event={"ID":"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644","Type":"ContainerStarted","Data":"819069d85ef064e579e5af755c633dd476b58ffdf64c5bd5572fbeff43899acb"}
Jul 21 13:08:02.734496 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:02.734439    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv" podStartSLOduration=1.734418535 podStartE2EDuration="1.734418535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:08:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:08:02.731859859 +0000 UTC m=+2420.516874825" watchObservedRunningTime="2026-07-21 13:08:02.734418535 +0000 UTC m=+2420.519433502"
Jul 21 13:08:11.813831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:11.813745    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:11.813831 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:11.813805    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:11.826476 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:11.826449    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:08:12.777360 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:08:12.777328    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:10:14.988314 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:14.988275    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"]
Jul 21 13:10:30.008703 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.008658    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv" podUID="2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" containerName="main" containerID="cri-o://30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e" gracePeriod=45
Jul 21 13:10:30.262679 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.262615    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:10:30.355058 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-home\") pod \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") "
Jul 21 13:10:30.355245 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355064    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-model-cache\") pod \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") "
Jul 21 13:10:30.355245 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355115    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tls-certs\") pod \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") "
Jul 21 13:10:30.355245 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355150    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tmp-dir\") pod \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") "
Jul 21 13:10:30.355245 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-dshm\") pod \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") "
Jul 21 13:10:30.355465 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355248    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bq8qz\" (UniqueName: \"kubernetes.io/projected/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-kube-api-access-bq8qz\") pod \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\" (UID: \"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644\") "
Jul 21 13:10:30.355465 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355337    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-home" (OuterVolumeSpecName: "home") pod "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" (UID: "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:30.355465 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355349    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-model-cache" (OuterVolumeSpecName: "model-cache") pod "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" (UID: "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:30.355634 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355498    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-home\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:10:30.355634 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355513    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-model-cache\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:10:30.355634 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.355493    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" (UID: "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:30.357399 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.357367    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" (UID: "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:10:30.357513 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.357407    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-dshm" (OuterVolumeSpecName: "dshm") pod "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" (UID: "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:10:30.357513 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.357447    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-kube-api-access-bq8qz" (OuterVolumeSpecName: "kube-api-access-bq8qz") pod "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" (UID: "2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644"). InnerVolumeSpecName "kube-api-access-bq8qz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:10:30.422643 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.422614    2577 generic.go:356] "Generic (PLEG): container finished" podID="2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" containerID="30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e" exitCode=0
Jul 21 13:10:30.422820 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.422661    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"
Jul 21 13:10:30.422820 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.422690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv" event={"ID":"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644","Type":"ContainerDied","Data":"30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e"}
Jul 21 13:10:30.422820 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.422729    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv" event={"ID":"2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644","Type":"ContainerDied","Data":"819069d85ef064e579e5af755c633dd476b58ffdf64c5bd5572fbeff43899acb"}
Jul 21 13:10:30.422820 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.422747    2577 scope.go:117] "RemoveContainer" containerID="30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e"
Jul 21 13:10:30.433029 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.433009    2577 scope.go:117] "RemoveContainer" containerID="30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e"
Jul 21 13:10:30.433288 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:10:30.433273    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e\": container with ID starting with 30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e not found: ID does not exist" containerID="30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e"
Jul 21 13:10:30.433341 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.433296    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e"} err="failed to get container status \"30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e\": rpc error: code = NotFound desc = could not find container \"30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e\": container with ID starting with 30c8d42c753e4823f5d8f9536dc5b1aa92ff099338efad23a4d1d1ff4537c19e not found: ID does not exist"
Jul 21 13:10:30.444255 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.444233    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"]
Jul 21 13:10:30.448184 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.448162    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/auth-test-model-b1b184-kserve-7cd997899f-7zhvv"]
Jul 21 13:10:30.456568 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.456550    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tmp-dir\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:10:30.456643 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.456569    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-dshm\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:10:30.456643 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.456579    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq8qz\" (UniqueName: \"kubernetes.io/projected/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-kube-api-access-bq8qz\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:10:30.456643 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.456589    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644-tls-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:10:30.834575 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:10:30.834542    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" path="/var/lib/kubelet/pods/2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644/volumes"
Jul 21 13:11:11.788584 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.788505    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"]
Jul 21 13:11:11.789054 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.788856    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" containerName="main"
Jul 21 13:11:11.789054 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.788867    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" containerName="main"
Jul 21 13:11:11.789054 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.788924    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa7d3e0-f4f7-4ff5-ac9d-4a7cce57e644" containerName="main"
Jul 21 13:11:11.792175 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.792157    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:11.795417 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.795383    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"openshift-service-ca.crt\""
Jul 21 13:11:11.795417 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.795397    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"default-dockercfg-4mxf7\""
Jul 21 13:11:11.795417 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.795405    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"kube-root-ca.crt\""
Jul 21 13:11:11.795663 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.795466    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"sub-test-model-b1b184-kserve-self-signed-certs\""
Jul 21 13:11:11.800984 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.800937    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"]
Jul 21 13:11:11.921684 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.921650    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-model-cache\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:11.921684 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.921683    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7946a792-4359-4f34-9b41-8ff434eaa5b5-tls-certs\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:11.921902 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.921702    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7cvx\" (UniqueName: \"kubernetes.io/projected/7946a792-4359-4f34-9b41-8ff434eaa5b5-kube-api-access-s7cvx\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:11.921902 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.921718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-home\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:11.921902 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.921748    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-tmp-dir\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:11.921902 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:11.921849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-dshm\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.022620 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.022592    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-model-cache\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.022620 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.022623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7946a792-4359-4f34-9b41-8ff434eaa5b5-tls-certs\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.022815 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.022640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7cvx\" (UniqueName: \"kubernetes.io/projected/7946a792-4359-4f34-9b41-8ff434eaa5b5-kube-api-access-s7cvx\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.022856 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.022835    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-home\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.022902 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.022885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-tmp-dir\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.023011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.022979    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-dshm\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.023446 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.023130    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-model-cache\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.023446 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.023253    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-home\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.024272 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.024249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-tmp-dir\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.025531 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.025500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7946a792-4359-4f34-9b41-8ff434eaa5b5-tls-certs\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.026082 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.026058    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-dshm\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.031782 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.031763    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7cvx\" (UniqueName: \"kubernetes.io/projected/7946a792-4359-4f34-9b41-8ff434eaa5b5-kube-api-access-s7cvx\") pod \"sub-test-model-b1b184-kserve-66487858f5-wk6mk\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") " pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.102378 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.102304    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:12.234320 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.234295    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"]
Jul 21 13:11:12.236544 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:11:12.236511    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7946a792_4359_4f34_9b41_8ff434eaa5b5.slice/crio-03ff5cc7645eb11294ea654ddc545148d9a6f494329762782e984dad96d801a6 WatchSource:0}: Error finding container 03ff5cc7645eb11294ea654ddc545148d9a6f494329762782e984dad96d801a6: Status 404 returned error can't find the container with id 03ff5cc7645eb11294ea654ddc545148d9a6f494329762782e984dad96d801a6
Jul 21 13:11:12.628582 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.628539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk" event={"ID":"7946a792-4359-4f34-9b41-8ff434eaa5b5","Type":"ContainerStarted","Data":"cabd7abfad225aef6cea08892846dcdfcca1eb210b072097244a6d6e1cd6a7c3"}
Jul 21 13:11:12.628582 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.628587    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk" event={"ID":"7946a792-4359-4f34-9b41-8ff434eaa5b5","Type":"ContainerStarted","Data":"03ff5cc7645eb11294ea654ddc545148d9a6f494329762782e984dad96d801a6"}
Jul 21 13:11:12.649049 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:12.649005    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk" podStartSLOduration=1.648991471 podStartE2EDuration="1.648991471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:11:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:11:12.64727522 +0000 UTC m=+2610.432290187" watchObservedRunningTime="2026-07-21 13:11:12.648991471 +0000 UTC m=+2610.434006436"
Jul 21 13:11:22.102592 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:22.102560    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:22.103006 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:22.102604    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:22.114605 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:22.114586    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:22.686198 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:22.686162    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:11:42.983527 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:11:42.983495    2577 scope.go:117] "RemoveContainer" containerID="1d3a62067bdc69fb230b824426c63686ee23be1245517a2912fd1b5d6962c353"
Jul 21 13:12:05.786094 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:05.786060    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"]
Jul 21 13:12:20.805721 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:20.805661    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk" podUID="7946a792-4359-4f34-9b41-8ff434eaa5b5" containerName="main" containerID="cri-o://cabd7abfad225aef6cea08892846dcdfcca1eb210b072097244a6d6e1cd6a7c3" gracePeriod=45
Jul 21 13:12:20.952522 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:20.952485    2577 generic.go:356] "Generic (PLEG): container finished" podID="7946a792-4359-4f34-9b41-8ff434eaa5b5" containerID="cabd7abfad225aef6cea08892846dcdfcca1eb210b072097244a6d6e1cd6a7c3" exitCode=0
Jul 21 13:12:20.952672 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:20.952557    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk" event={"ID":"7946a792-4359-4f34-9b41-8ff434eaa5b5","Type":"ContainerDied","Data":"cabd7abfad225aef6cea08892846dcdfcca1eb210b072097244a6d6e1cd6a7c3"}
Jul 21 13:12:21.063700 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.063644    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:12:21.107439 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107412    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-home\") pod \"7946a792-4359-4f34-9b41-8ff434eaa5b5\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") "
Jul 21 13:12:21.107576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107452    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-model-cache\") pod \"7946a792-4359-4f34-9b41-8ff434eaa5b5\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") "
Jul 21 13:12:21.107576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-tmp-dir\") pod \"7946a792-4359-4f34-9b41-8ff434eaa5b5\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") "
Jul 21 13:12:21.107576 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107512    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7946a792-4359-4f34-9b41-8ff434eaa5b5-tls-certs\") pod \"7946a792-4359-4f34-9b41-8ff434eaa5b5\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") "
Jul 21 13:12:21.107751 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107568    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-dshm\") pod \"7946a792-4359-4f34-9b41-8ff434eaa5b5\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") "
Jul 21 13:12:21.107751 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107644    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7cvx\" (UniqueName: \"kubernetes.io/projected/7946a792-4359-4f34-9b41-8ff434eaa5b5-kube-api-access-s7cvx\") pod \"7946a792-4359-4f34-9b41-8ff434eaa5b5\" (UID: \"7946a792-4359-4f34-9b41-8ff434eaa5b5\") "
Jul 21 13:12:21.107751 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107688    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-home" (OuterVolumeSpecName: "home") pod "7946a792-4359-4f34-9b41-8ff434eaa5b5" (UID: "7946a792-4359-4f34-9b41-8ff434eaa5b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:12:21.107882 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107761    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7946a792-4359-4f34-9b41-8ff434eaa5b5" (UID: "7946a792-4359-4f34-9b41-8ff434eaa5b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:12:21.107882 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107775    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "7946a792-4359-4f34-9b41-8ff434eaa5b5" (UID: "7946a792-4359-4f34-9b41-8ff434eaa5b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:12:21.107985 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107909    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-home\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:12:21.107985 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107922    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-model-cache\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:12:21.107985 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.107933    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-tmp-dir\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:12:21.109713 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.109690    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-dshm" (OuterVolumeSpecName: "dshm") pod "7946a792-4359-4f34-9b41-8ff434eaa5b5" (UID: "7946a792-4359-4f34-9b41-8ff434eaa5b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:12:21.109713 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.109694    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7946a792-4359-4f34-9b41-8ff434eaa5b5-kube-api-access-s7cvx" (OuterVolumeSpecName: "kube-api-access-s7cvx") pod "7946a792-4359-4f34-9b41-8ff434eaa5b5" (UID: "7946a792-4359-4f34-9b41-8ff434eaa5b5"). InnerVolumeSpecName "kube-api-access-s7cvx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:12:21.109864 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.109737    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7946a792-4359-4f34-9b41-8ff434eaa5b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7946a792-4359-4f34-9b41-8ff434eaa5b5" (UID: "7946a792-4359-4f34-9b41-8ff434eaa5b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:12:21.208533 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.208499    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7cvx\" (UniqueName: \"kubernetes.io/projected/7946a792-4359-4f34-9b41-8ff434eaa5b5-kube-api-access-s7cvx\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:12:21.208533 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.208526    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7946a792-4359-4f34-9b41-8ff434eaa5b5-tls-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:12:21.208533 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.208540    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7946a792-4359-4f34-9b41-8ff434eaa5b5-dshm\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:12:21.958800 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.958765    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk" event={"ID":"7946a792-4359-4f34-9b41-8ff434eaa5b5","Type":"ContainerDied","Data":"03ff5cc7645eb11294ea654ddc545148d9a6f494329762782e984dad96d801a6"}
Jul 21 13:12:21.959263 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.958814    2577 scope.go:117] "RemoveContainer" containerID="cabd7abfad225aef6cea08892846dcdfcca1eb210b072097244a6d6e1cd6a7c3"
Jul 21 13:12:21.959263 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.958782    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"
Jul 21 13:12:21.979714 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.979690    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"]
Jul 21 13:12:21.984052 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:21.984030    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/sub-test-model-b1b184-kserve-66487858f5-wk6mk"]
Jul 21 13:12:22.832595 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:12:22.832559    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7946a792-4359-4f34-9b41-8ff434eaa5b5" path="/var/lib/kubelet/pods/7946a792-4359-4f34-9b41-8ff434eaa5b5/volumes"
Jul 21 13:13:18.885985 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.885934    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"]
Jul 21 13:13:18.886440 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.886292    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7946a792-4359-4f34-9b41-8ff434eaa5b5" containerName="main"
Jul 21 13:13:18.886440 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.886303    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7946a792-4359-4f34-9b41-8ff434eaa5b5" containerName="main"
Jul 21 13:13:18.886440 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.886359    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7946a792-4359-4f34-9b41-8ff434eaa5b5" containerName="main"
Jul 21 13:13:18.889184 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.889168    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:18.891923 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.891900    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"openshift-service-ca.crt\""
Jul 21 13:13:18.892081 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.892066    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"default-dockercfg-4mxf7\""
Jul 21 13:13:18.892850 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.892785    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"kube-root-ca.crt\""
Jul 21 13:13:18.892850 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.892800    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-b1b184\"/\"rate-test-model-b1b184-kserve-self-signed-certs\""
Jul 21 13:13:18.903011 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.902986    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"]
Jul 21 13:13:18.986193 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.986159    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-home\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:18.986353 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.986196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-model-cache\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:18.986353 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.986220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-tmp-dir\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:18.986353 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.986243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b923b690-b596-47fb-9ec8-819603c439b5-tls-certs\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:18.986353 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.986280    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmpf6\" (UniqueName: \"kubernetes.io/projected/b923b690-b596-47fb-9ec8-819603c439b5-kube-api-access-kmpf6\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:18.986526 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:18.986406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-dshm\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.087784 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.087746    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-dshm\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.087941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.087793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-home\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.087941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.087809    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-model-cache\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.087941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.087829    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-tmp-dir\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.087941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.087854    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b923b690-b596-47fb-9ec8-819603c439b5-tls-certs\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.087941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.087890    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmpf6\" (UniqueName: \"kubernetes.io/projected/b923b690-b596-47fb-9ec8-819603c439b5-kube-api-access-kmpf6\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.088264 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.088189    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-home\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.088264 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.088242    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-model-cache\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.088370 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.088313    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-tmp-dir\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.090190 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.090166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-dshm\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.090492 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.090469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b923b690-b596-47fb-9ec8-819603c439b5-tls-certs\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.098135 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.098112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmpf6\" (UniqueName: \"kubernetes.io/projected/b923b690-b596-47fb-9ec8-819603c439b5-kube-api-access-kmpf6\") pod \"rate-test-model-b1b184-kserve-f87b7d58c-2rd4t\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") " pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.199209 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.199142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:19.335596 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:19.335483    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"]
Jul 21 13:13:19.338222 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:13:19.338190    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb923b690_b596_47fb_9ec8_819603c439b5.slice/crio-6f3695fd691ce7807be1478119ae8af91099a37edbfeb216d833d48318cce7e7 WatchSource:0}: Error finding container 6f3695fd691ce7807be1478119ae8af91099a37edbfeb216d833d48318cce7e7: Status 404 returned error can't find the container with id 6f3695fd691ce7807be1478119ae8af91099a37edbfeb216d833d48318cce7e7
Jul 21 13:13:20.229792 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:20.229753    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t" event={"ID":"b923b690-b596-47fb-9ec8-819603c439b5","Type":"ContainerStarted","Data":"d734d97751fdf9231c91d41f005d5fa6f64e6ba6c27578d9faf0be0b5df1e6af"}
Jul 21 13:13:20.230202 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:20.229802    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t" event={"ID":"b923b690-b596-47fb-9ec8-819603c439b5","Type":"ContainerStarted","Data":"6f3695fd691ce7807be1478119ae8af91099a37edbfeb216d833d48318cce7e7"}
Jul 21 13:13:20.247636 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:20.247589    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t" podStartSLOduration=2.247573714 podStartE2EDuration="2.247573714s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:13:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:13:20.247280431 +0000 UTC m=+2738.032295397" watchObservedRunningTime="2026-07-21 13:13:20.247573714 +0000 UTC m=+2738.032588753"
Jul 21 13:13:29.199512 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:29.199478    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:29.199512 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:29.199521    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:29.212023 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:29.212002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:13:29.282491 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:13:29.282464    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:14:04.292438 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:04.292352    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"]
Jul 21 13:14:05.276467 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.276436    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139725296028080"]
Jul 21 13:14:05.279863 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.279844    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296028080"
Jul 21 13:14:05.282276 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.282245    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rjjv6\""
Jul 21 13:14:05.284590 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.284560    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296028080"]
Jul 21 13:14:05.380487 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.380450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mjtv\" (UniqueName: \"kubernetes.io/projected/35413586-cb18-4e6e-8cc5-4d979e2a4092-kube-api-access-8mjtv\") pod \"test-curl-7509-139725296028080\" (UID: \"35413586-cb18-4e6e-8cc5-4d979e2a4092\") " pod="opendatahub/test-curl-7509-139725296028080"
Jul 21 13:14:05.481362 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.481334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mjtv\" (UniqueName: \"kubernetes.io/projected/35413586-cb18-4e6e-8cc5-4d979e2a4092-kube-api-access-8mjtv\") pod \"test-curl-7509-139725296028080\" (UID: \"35413586-cb18-4e6e-8cc5-4d979e2a4092\") " pod="opendatahub/test-curl-7509-139725296028080"
Jul 21 13:14:05.489499 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.489476    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mjtv\" (UniqueName: \"kubernetes.io/projected/35413586-cb18-4e6e-8cc5-4d979e2a4092-kube-api-access-8mjtv\") pod \"test-curl-7509-139725296028080\" (UID: \"35413586-cb18-4e6e-8cc5-4d979e2a4092\") " pod="opendatahub/test-curl-7509-139725296028080"
Jul 21 13:14:05.589806 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.589715    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296028080"
Jul 21 13:14:05.720604 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.720581    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296028080"]
Jul 21 13:14:05.722724 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:14:05.722695    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod35413586_cb18_4e6e_8cc5_4d979e2a4092.slice/crio-399dc4f191ab348a50f6799ea1adf480ae149791de078de5f23a42b39cc8b170 WatchSource:0}: Error finding container 399dc4f191ab348a50f6799ea1adf480ae149791de078de5f23a42b39cc8b170: Status 404 returned error can't find the container with id 399dc4f191ab348a50f6799ea1adf480ae149791de078de5f23a42b39cc8b170
Jul 21 13:14:05.724131 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:05.724110    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 21 13:14:06.444149 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:06.444113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296028080" event={"ID":"35413586-cb18-4e6e-8cc5-4d979e2a4092","Type":"ContainerStarted","Data":"399dc4f191ab348a50f6799ea1adf480ae149791de078de5f23a42b39cc8b170"}
Jul 21 13:14:07.449660 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:07.449622    2577 generic.go:356] "Generic (PLEG): container finished" podID="35413586-cb18-4e6e-8cc5-4d979e2a4092" containerID="1f394cdc797786ab0b14c4829f0c954dcb4081467d1f8080d82713113dc1dc84" exitCode=0
Jul 21 13:14:07.450077 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:07.449700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296028080" event={"ID":"35413586-cb18-4e6e-8cc5-4d979e2a4092","Type":"ContainerDied","Data":"1f394cdc797786ab0b14c4829f0c954dcb4081467d1f8080d82713113dc1dc84"}
Jul 21 13:14:08.584020 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.583996    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296028080"
Jul 21 13:14:08.605090 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.605066    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8mjtv\" (UniqueName: \"kubernetes.io/projected/35413586-cb18-4e6e-8cc5-4d979e2a4092-kube-api-access-8mjtv\") pod \"35413586-cb18-4e6e-8cc5-4d979e2a4092\" (UID: \"35413586-cb18-4e6e-8cc5-4d979e2a4092\") "
Jul 21 13:14:08.607266 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.607232    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35413586-cb18-4e6e-8cc5-4d979e2a4092-kube-api-access-8mjtv" (OuterVolumeSpecName: "kube-api-access-8mjtv") pod "35413586-cb18-4e6e-8cc5-4d979e2a4092" (UID: "35413586-cb18-4e6e-8cc5-4d979e2a4092"). InnerVolumeSpecName "kube-api-access-8mjtv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:14:08.632632 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.632609    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139725296028080_35413586-cb18-4e6e-8cc5-4d979e2a4092/test-curl-7509-139725296028080/0.log"
Jul 21 13:14:08.669793 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.669767    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139725296028080"]
Jul 21 13:14:08.676451 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.676430    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139725296028080"]
Jul 21 13:14:08.705814 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.705748    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8mjtv\" (UniqueName: \"kubernetes.io/projected/35413586-cb18-4e6e-8cc5-4d979e2a4092-kube-api-access-8mjtv\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:08.832569 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:08.832535    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35413586-cb18-4e6e-8cc5-4d979e2a4092" path="/var/lib/kubelet/pods/35413586-cb18-4e6e-8cc5-4d979e2a4092/volumes"
Jul 21 13:14:09.460858 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:09.460830    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296028080"
Jul 21 13:14:09.461046 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:09.460874    2577 scope.go:117] "RemoveContainer" containerID="1f394cdc797786ab0b14c4829f0c954dcb4081467d1f8080d82713113dc1dc84"
Jul 21 13:14:18.130697 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.130662    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:18.131112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.131040    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35413586-cb18-4e6e-8cc5-4d979e2a4092" containerName="test-curl-7509-139725296028080"
Jul 21 13:14:18.131112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.131053    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="35413586-cb18-4e6e-8cc5-4d979e2a4092" containerName="test-curl-7509-139725296028080"
Jul 21 13:14:18.131188 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.131114    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="35413586-cb18-4e6e-8cc5-4d979e2a4092" containerName="test-curl-7509-139725296028080"
Jul 21 13:14:18.135440 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.135424    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:18.137895 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.137868    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-rjjv6\""
Jul 21 13:14:18.138820 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.138792    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:18.186484 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.186463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h28fz\" (UniqueName: \"kubernetes.io/projected/3bc9c875-f525-4599-a0e6-1d1b80800d70-kube-api-access-h28fz\") pod \"test-curl-7509-139725296089248\" (UID: \"3bc9c875-f525-4599-a0e6-1d1b80800d70\") " pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:18.287667 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.287635    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h28fz\" (UniqueName: \"kubernetes.io/projected/3bc9c875-f525-4599-a0e6-1d1b80800d70-kube-api-access-h28fz\") pod \"test-curl-7509-139725296089248\" (UID: \"3bc9c875-f525-4599-a0e6-1d1b80800d70\") " pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:18.297099 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.297069    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h28fz\" (UniqueName: \"kubernetes.io/projected/3bc9c875-f525-4599-a0e6-1d1b80800d70-kube-api-access-h28fz\") pod \"test-curl-7509-139725296089248\" (UID: \"3bc9c875-f525-4599-a0e6-1d1b80800d70\") " pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:18.446816 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.446737    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:18.581213 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:18.581135    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:18.583788 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:14:18.583760    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3bc9c875_f525_4599_a0e6_1d1b80800d70.slice/crio-3cdd478f5a940c3c1e5957b56ce578f6d0ec069e4709ad64e813f15e3f6b9709 WatchSource:0}: Error finding container 3cdd478f5a940c3c1e5957b56ce578f6d0ec069e4709ad64e813f15e3f6b9709: Status 404 returned error can't find the container with id 3cdd478f5a940c3c1e5957b56ce578f6d0ec069e4709ad64e813f15e3f6b9709
Jul 21 13:14:19.314147 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.314100    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t" podUID="b923b690-b596-47fb-9ec8-819603c439b5" containerName="main" containerID="cri-o://d734d97751fdf9231c91d41f005d5fa6f64e6ba6c27578d9faf0be0b5df1e6af" gracePeriod=45
Jul 21 13:14:19.513060 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.513027    2577 generic.go:356] "Generic (PLEG): container finished" podID="b923b690-b596-47fb-9ec8-819603c439b5" containerID="d734d97751fdf9231c91d41f005d5fa6f64e6ba6c27578d9faf0be0b5df1e6af" exitCode=0
Jul 21 13:14:19.513198 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.513096    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t" event={"ID":"b923b690-b596-47fb-9ec8-819603c439b5","Type":"ContainerDied","Data":"d734d97751fdf9231c91d41f005d5fa6f64e6ba6c27578d9faf0be0b5df1e6af"}
Jul 21 13:14:19.514532 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.514509    2577 generic.go:356] "Generic (PLEG): container finished" podID="3bc9c875-f525-4599-a0e6-1d1b80800d70" containerID="9504fb6e4b335464c347f918ce3d55280c53e70c6579bff63e6081cc4a7924e9" exitCode=0
Jul 21 13:14:19.514647 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.514606    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296089248" event={"ID":"3bc9c875-f525-4599-a0e6-1d1b80800d70","Type":"ContainerDied","Data":"9504fb6e4b335464c347f918ce3d55280c53e70c6579bff63e6081cc4a7924e9"}
Jul 21 13:14:19.514688 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.514647    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296089248" event={"ID":"3bc9c875-f525-4599-a0e6-1d1b80800d70","Type":"ContainerStarted","Data":"3cdd478f5a940c3c1e5957b56ce578f6d0ec069e4709ad64e813f15e3f6b9709"}
Jul 21 13:14:19.555836 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.555785    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:14:19.597886 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.597866    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-tmp-dir\") pod \"b923b690-b596-47fb-9ec8-819603c439b5\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") "
Jul 21 13:14:19.598020 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.597932    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-model-cache\") pod \"b923b690-b596-47fb-9ec8-819603c439b5\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") "
Jul 21 13:14:19.598020 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.597975    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b923b690-b596-47fb-9ec8-819603c439b5-tls-certs\") pod \"b923b690-b596-47fb-9ec8-819603c439b5\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") "
Jul 21 13:14:19.598020 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.598006    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-home\") pod \"b923b690-b596-47fb-9ec8-819603c439b5\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") "
Jul 21 13:14:19.598192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.598028    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-dshm\") pod \"b923b690-b596-47fb-9ec8-819603c439b5\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") "
Jul 21 13:14:19.598192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.598063    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kmpf6\" (UniqueName: \"kubernetes.io/projected/b923b690-b596-47fb-9ec8-819603c439b5-kube-api-access-kmpf6\") pod \"b923b690-b596-47fb-9ec8-819603c439b5\" (UID: \"b923b690-b596-47fb-9ec8-819603c439b5\") "
Jul 21 13:14:19.598192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.598126    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b923b690-b596-47fb-9ec8-819603c439b5" (UID: "b923b690-b596-47fb-9ec8-819603c439b5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:19.598347 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.598237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-model-cache" (OuterVolumeSpecName: "model-cache") pod "b923b690-b596-47fb-9ec8-819603c439b5" (UID: "b923b690-b596-47fb-9ec8-819603c439b5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:19.598347 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.598252    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-home" (OuterVolumeSpecName: "home") pod "b923b690-b596-47fb-9ec8-819603c439b5" (UID: "b923b690-b596-47fb-9ec8-819603c439b5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:19.598347 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.598289    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-tmp-dir\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:19.600168 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.600145    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b923b690-b596-47fb-9ec8-819603c439b5-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b923b690-b596-47fb-9ec8-819603c439b5" (UID: "b923b690-b596-47fb-9ec8-819603c439b5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 21 13:14:19.600237 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.600149    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b923b690-b596-47fb-9ec8-819603c439b5-kube-api-access-kmpf6" (OuterVolumeSpecName: "kube-api-access-kmpf6") pod "b923b690-b596-47fb-9ec8-819603c439b5" (UID: "b923b690-b596-47fb-9ec8-819603c439b5"). InnerVolumeSpecName "kube-api-access-kmpf6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:14:19.600237 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.600145    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-dshm" (OuterVolumeSpecName: "dshm") pod "b923b690-b596-47fb-9ec8-819603c439b5" (UID: "b923b690-b596-47fb-9ec8-819603c439b5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:19.698872 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.698841    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-model-cache\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:19.698872 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.698867    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b923b690-b596-47fb-9ec8-819603c439b5-tls-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:19.698872 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.698877    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-home\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:19.699132 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.698884    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b923b690-b596-47fb-9ec8-819603c439b5-dshm\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:19.699132 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:19.698893    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kmpf6\" (UniqueName: \"kubernetes.io/projected/b923b690-b596-47fb-9ec8-819603c439b5-kube-api-access-kmpf6\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:20.520227 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.520197    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"
Jul 21 13:14:20.520679 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.520233    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t" event={"ID":"b923b690-b596-47fb-9ec8-819603c439b5","Type":"ContainerDied","Data":"6f3695fd691ce7807be1478119ae8af91099a37edbfeb216d833d48318cce7e7"}
Jul 21 13:14:20.520679 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.520284    2577 scope.go:117] "RemoveContainer" containerID="d734d97751fdf9231c91d41f005d5fa6f64e6ba6c27578d9faf0be0b5df1e6af"
Jul 21 13:14:20.558688 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.558656    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"]
Jul 21 13:14:20.575441 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.575414    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-b1b184/rate-test-model-b1b184-kserve-f87b7d58c-2rd4t"]
Jul 21 13:14:20.655465 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.655442    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:20.678981 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.678930    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139725296089248_3bc9c875-f525-4599-a0e6-1d1b80800d70/test-curl-7509-139725296089248/0.log"
Jul 21 13:14:20.703829 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.703802    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:20.707192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.707168    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h28fz\" (UniqueName: \"kubernetes.io/projected/3bc9c875-f525-4599-a0e6-1d1b80800d70-kube-api-access-h28fz\") pod \"3bc9c875-f525-4599-a0e6-1d1b80800d70\" (UID: \"3bc9c875-f525-4599-a0e6-1d1b80800d70\") "
Jul 21 13:14:20.707578 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.707557    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:20.708971 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.708938    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bc9c875-f525-4599-a0e6-1d1b80800d70-kube-api-access-h28fz" (OuterVolumeSpecName: "kube-api-access-h28fz") pod "3bc9c875-f525-4599-a0e6-1d1b80800d70" (UID: "3bc9c875-f525-4599-a0e6-1d1b80800d70"). InnerVolumeSpecName "kube-api-access-h28fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:14:20.808333 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.808276    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h28fz\" (UniqueName: \"kubernetes.io/projected/3bc9c875-f525-4599-a0e6-1d1b80800d70-kube-api-access-h28fz\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:20.832271 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.832239    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bc9c875-f525-4599-a0e6-1d1b80800d70" path="/var/lib/kubelet/pods/3bc9c875-f525-4599-a0e6-1d1b80800d70/volumes"
Jul 21 13:14:20.832628 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.832613    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b923b690-b596-47fb-9ec8-819603c439b5" path="/var/lib/kubelet/pods/b923b690-b596-47fb-9ec8-819603c439b5/volumes"
Jul 21 13:14:20.833441 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.833419    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:20.833791 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.833774    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b923b690-b596-47fb-9ec8-819603c439b5" containerName="main"
Jul 21 13:14:20.833868 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.833794    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b923b690-b596-47fb-9ec8-819603c439b5" containerName="main"
Jul 21 13:14:20.833868 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.833821    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bc9c875-f525-4599-a0e6-1d1b80800d70" containerName="test-curl-7509-139725296089248"
Jul 21 13:14:20.833868 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.833831    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bc9c875-f525-4599-a0e6-1d1b80800d70" containerName="test-curl-7509-139725296089248"
Jul 21 13:14:20.834039 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.833919    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bc9c875-f525-4599-a0e6-1d1b80800d70" containerName="test-curl-7509-139725296089248"
Jul 21 13:14:20.834039 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.833931    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b923b690-b596-47fb-9ec8-819603c439b5" containerName="main"
Jul 21 13:14:20.837015 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.836997    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:20.841710 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.841686    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:20.909151 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:20.909125    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xhpph\" (UniqueName: \"kubernetes.io/projected/8de09db8-1ecc-43ef-bb06-12387d2ac441-kube-api-access-xhpph\") pod \"test-curl-7509-139725296089248\" (UID: \"8de09db8-1ecc-43ef-bb06-12387d2ac441\") " pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:21.010423 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:21.010393    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xhpph\" (UniqueName: \"kubernetes.io/projected/8de09db8-1ecc-43ef-bb06-12387d2ac441-kube-api-access-xhpph\") pod \"test-curl-7509-139725296089248\" (UID: \"8de09db8-1ecc-43ef-bb06-12387d2ac441\") " pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:21.022951 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:21.022927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xhpph\" (UniqueName: \"kubernetes.io/projected/8de09db8-1ecc-43ef-bb06-12387d2ac441-kube-api-access-xhpph\") pod \"test-curl-7509-139725296089248\" (UID: \"8de09db8-1ecc-43ef-bb06-12387d2ac441\") " pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:21.149150 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:21.149068    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:21.278219 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:21.278196    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:21.280380 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:14:21.280351    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8de09db8_1ecc_43ef_bb06_12387d2ac441.slice/crio-97869b4500f1b34d3b8e33e1edf5d7baf99f597a3321e2b02e4544c50d00d2b7 WatchSource:0}: Error finding container 97869b4500f1b34d3b8e33e1edf5d7baf99f597a3321e2b02e4544c50d00d2b7: Status 404 returned error can't find the container with id 97869b4500f1b34d3b8e33e1edf5d7baf99f597a3321e2b02e4544c50d00d2b7
Jul 21 13:14:21.527559 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:21.527531    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:21.528024 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:21.527541    2577 scope.go:117] "RemoveContainer" containerID="9504fb6e4b335464c347f918ce3d55280c53e70c6579bff63e6081cc4a7924e9"
Jul 21 13:14:21.528788 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:21.528764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296089248" event={"ID":"8de09db8-1ecc-43ef-bb06-12387d2ac441","Type":"ContainerStarted","Data":"97869b4500f1b34d3b8e33e1edf5d7baf99f597a3321e2b02e4544c50d00d2b7"}
Jul 21 13:14:22.535674 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:22.535640    2577 generic.go:356] "Generic (PLEG): container finished" podID="8de09db8-1ecc-43ef-bb06-12387d2ac441" containerID="c261d7ac46ef72ac35580c94a62b12badeec515306bb1104868ca29b1960c751" exitCode=0
Jul 21 13:14:22.536122 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:22.535690    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7509-139725296089248" event={"ID":"8de09db8-1ecc-43ef-bb06-12387d2ac441","Type":"ContainerDied","Data":"c261d7ac46ef72ac35580c94a62b12badeec515306bb1104868ca29b1960c751"}
Jul 21 13:14:23.671974 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.671938    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:23.708314 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.708283    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7509-139725296089248_8de09db8-1ecc-43ef-bb06-12387d2ac441/test-curl-7509-139725296089248/0.log"
Jul 21 13:14:23.740154 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.740131    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xhpph\" (UniqueName: \"kubernetes.io/projected/8de09db8-1ecc-43ef-bb06-12387d2ac441-kube-api-access-xhpph\") pod \"8de09db8-1ecc-43ef-bb06-12387d2ac441\" (UID: \"8de09db8-1ecc-43ef-bb06-12387d2ac441\") "
Jul 21 13:14:23.742225 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.742200    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8de09db8-1ecc-43ef-bb06-12387d2ac441-kube-api-access-xhpph" (OuterVolumeSpecName: "kube-api-access-xhpph") pod "8de09db8-1ecc-43ef-bb06-12387d2ac441" (UID: "8de09db8-1ecc-43ef-bb06-12387d2ac441"). InnerVolumeSpecName "kube-api-access-xhpph". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:14:23.760697 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.760674    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:23.767377 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.767356    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7509-139725296089248"]
Jul 21 13:14:23.840793 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.840738    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xhpph\" (UniqueName: \"kubernetes.io/projected/8de09db8-1ecc-43ef-bb06-12387d2ac441-kube-api-access-xhpph\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:23.970709 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.970680    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"]
Jul 21 13:14:23.971048 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.971034    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8de09db8-1ecc-43ef-bb06-12387d2ac441" containerName="test-curl-7509-139725296089248"
Jul 21 13:14:23.971095 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.971049    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8de09db8-1ecc-43ef-bb06-12387d2ac441" containerName="test-curl-7509-139725296089248"
Jul 21 13:14:23.971141 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.971131    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="8de09db8-1ecc-43ef-bb06-12387d2ac441" containerName="test-curl-7509-139725296089248"
Jul 21 13:14:23.974081 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.974061    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"
Jul 21 13:14:23.976594 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.976572    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 21 13:14:23.976695 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.976618    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jq8px\""
Jul 21 13:14:23.983652 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:23.983625    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"]
Jul 21 13:14:24.042832 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.042795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db7bafd-dca6-417b-9cca-6509def49c0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs\" (UID: \"2db7bafd-dca6-417b-9cca-6509def49c0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"
Jul 21 13:14:24.144151 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.144119    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db7bafd-dca6-417b-9cca-6509def49c0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs\" (UID: \"2db7bafd-dca6-417b-9cca-6509def49c0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"
Jul 21 13:14:24.144712 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.144691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db7bafd-dca6-417b-9cca-6509def49c0f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs\" (UID: \"2db7bafd-dca6-417b-9cca-6509def49c0f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"
Jul 21 13:14:24.285676 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.285646    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"
Jul 21 13:14:24.545603 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.545579    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:24.545603 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.545587    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97869b4500f1b34d3b8e33e1edf5d7baf99f597a3321e2b02e4544c50d00d2b7"
Jul 21 13:14:24.555864 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:14:24.555836    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7509-139725296089248\" is forbidden: User \"system:node:ip-10-0-143-161.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-143-161.ec2.internal' and this object" podUID="8de09db8-1ecc-43ef-bb06-12387d2ac441" pod="opendatahub/test-curl-7509-139725296089248"
Jul 21 13:14:24.624366 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.624343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"]
Jul 21 13:14:24.626492 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:14:24.626465    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2db7bafd_dca6_417b_9cca_6509def49c0f.slice/crio-470931fc3703530c368bbbf1f8c125d07d295853cc827a11353ba469f978f033 WatchSource:0}: Error finding container 470931fc3703530c368bbbf1f8c125d07d295853cc827a11353ba469f978f033: Status 404 returned error can't find the container with id 470931fc3703530c368bbbf1f8c125d07d295853cc827a11353ba469f978f033
Jul 21 13:14:24.832663 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:24.832590    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8de09db8-1ecc-43ef-bb06-12387d2ac441" path="/var/lib/kubelet/pods/8de09db8-1ecc-43ef-bb06-12387d2ac441/volumes"
Jul 21 13:14:25.551794 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:25.551763    2577 generic.go:356] "Generic (PLEG): container finished" podID="2db7bafd-dca6-417b-9cca-6509def49c0f" containerID="be755aa6bf919a27d7730b2125121baca299ea79aa8516e50f2c5b3fb0f0549c" exitCode=0
Jul 21 13:14:25.551918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:25.551856    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs" event={"ID":"2db7bafd-dca6-417b-9cca-6509def49c0f","Type":"ContainerDied","Data":"be755aa6bf919a27d7730b2125121baca299ea79aa8516e50f2c5b3fb0f0549c"}
Jul 21 13:14:25.551918 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:25.551908    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs" event={"ID":"2db7bafd-dca6-417b-9cca-6509def49c0f","Type":"ContainerStarted","Data":"470931fc3703530c368bbbf1f8c125d07d295853cc827a11353ba469f978f033"}
Jul 21 13:14:26.691095 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:26.691066    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"
Jul 21 13:14:26.765824 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:26.765797    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db7bafd-dca6-417b-9cca-6509def49c0f-maas-api-service-ca\") pod \"2db7bafd-dca6-417b-9cca-6509def49c0f\" (UID: \"2db7bafd-dca6-417b-9cca-6509def49c0f\") "
Jul 21 13:14:26.766159 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:26.766138    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2db7bafd-dca6-417b-9cca-6509def49c0f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2db7bafd-dca6-417b-9cca-6509def49c0f" (UID: "2db7bafd-dca6-417b-9cca-6509def49c0f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:14:26.867385 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:26.867348    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2db7bafd-dca6-417b-9cca-6509def49c0f-maas-api-service-ca\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:27.561933 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:27.561908    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"
Jul 21 13:14:27.562085 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:27.561933    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs" event={"ID":"2db7bafd-dca6-417b-9cca-6509def49c0f","Type":"ContainerDied","Data":"470931fc3703530c368bbbf1f8c125d07d295853cc827a11353ba469f978f033"}
Jul 21 13:14:27.562085 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:27.561991    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="470931fc3703530c368bbbf1f8c125d07d295853cc827a11353ba469f978f033"
Jul 21 13:14:39.076804 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:39.076769    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"]
Jul 21 13:14:39.079321 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:39.079299    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-3cf59c-lpwvs"]
Jul 21 13:14:40.833932 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:40.833901    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2db7bafd-dca6-417b-9cca-6509def49c0f" path="/var/lib/kubelet/pods/2db7bafd-dca6-417b-9cca-6509def49c0f/volumes"
Jul 21 13:14:45.716968 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:45.716931    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"]
Jul 21 13:14:45.717413 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:45.717151    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy" containerID="cri-o://0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d" gracePeriod=30
Jul 21 13:14:50.972882 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:50.972855    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:14:51.073022 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.072978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8hc52\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-kube-api-access-8hc52\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073169 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073099    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-token\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073169 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073121    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-podinfo\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073169 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073160    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-socket\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073337 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073181    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-data\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073337 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073213    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-credential-socket\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073337 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073250    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istiod-ca-cert\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073337 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073295    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-envoy\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073337 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073331    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-certs\") pod \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\" (UID: \"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7\") "
Jul 21 13:14:51.073597 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073512    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:51.073668 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073629    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:14:51.073786 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073764    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:51.073898 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.073874    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:51.074199 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.074175    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-data" (OuterVolumeSpecName: "istio-data") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:51.075425 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.075398    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:14:51.075524 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.075433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:14:51.075608 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.075591    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-kube-api-access-8hc52" (OuterVolumeSpecName: "kube-api-access-8hc52") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "kube-api-access-8hc52". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:14:51.075647 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.075607    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-token" (OuterVolumeSpecName: "istio-token") pod "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" (UID: "b66cccae-c3e7-4e6f-a150-200ad5c5b8e7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:14:51.173921 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173891    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.173921 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173917    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.174112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173925    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8hc52\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-kube-api-access-8hc52\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.174112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173935    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.174112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173946    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.174112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173976    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.174112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173985    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.174112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.173994    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.174112 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.174002    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:14:51.672666 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.672627    2577 generic.go:356] "Generic (PLEG): container finished" podID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerID="0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d" exitCode=0
Jul 21 13:14:51.672840 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.672682    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"
Jul 21 13:14:51.672840 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.672714    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" event={"ID":"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7","Type":"ContainerDied","Data":"0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d"}
Jul 21 13:14:51.672840 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.672770    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx" event={"ID":"b66cccae-c3e7-4e6f-a150-200ad5c5b8e7","Type":"ContainerDied","Data":"5226056befaecb636ca3588c3a8daf4124f235cf13a133750b01be9d210782d6"}
Jul 21 13:14:51.672840 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.672793    2577 scope.go:117] "RemoveContainer" containerID="0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d"
Jul 21 13:14:51.682078 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.682060    2577 scope.go:117] "RemoveContainer" containerID="792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1"
Jul 21 13:14:51.690123 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.690101    2577 scope.go:117] "RemoveContainer" containerID="0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d"
Jul 21 13:14:51.690364 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:14:51.690348    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d\": container with ID starting with 0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d not found: ID does not exist" containerID="0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d"
Jul 21 13:14:51.690415 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.690372    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d"} err="failed to get container status \"0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d\": rpc error: code = NotFound desc = could not find container \"0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d\": container with ID starting with 0158b587195ae7a53403960b99e3794b6f1643fc876bc0c428813aedbfa4bc8d not found: ID does not exist"
Jul 21 13:14:51.690415 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.690387    2577 scope.go:117] "RemoveContainer" containerID="792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1"
Jul 21 13:14:51.690594 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:14:51.690578    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1\": container with ID starting with 792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1 not found: ID does not exist" containerID="792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1"
Jul 21 13:14:51.690644 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.690598    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1"} err="failed to get container status \"792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1\": rpc error: code = NotFound desc = could not find container \"792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1\": container with ID starting with 792d5cbed5b7b825d036fb806a4103719465d58d1fa4121ea1dd5a648e1858c1 not found: ID does not exist"
Jul 21 13:14:51.695429 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.695405    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"]
Jul 21 13:14:51.708731 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:51.708709    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-3cf59c-openshift-default-6d58c465f5-vhgjx"]
Jul 21 13:14:52.832012 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:14:52.831979    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" path="/var/lib/kubelet/pods/b66cccae-c3e7-4e6f-a150-200ad5c5b8e7/volumes"
Jul 21 13:15:00.179895 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.179860    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd"]
Jul 21 13:15:00.180295 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.180236    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy"
Jul 21 13:15:00.180295 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.180250    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy"
Jul 21 13:15:00.180295 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.180261    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2db7bafd-dca6-417b-9cca-6509def49c0f" containerName="revoke-keys"
Jul 21 13:15:00.180295 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.180267    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2db7bafd-dca6-417b-9cca-6509def49c0f" containerName="revoke-keys"
Jul 21 13:15:00.180435 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.180333    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy"
Jul 21 13:15:00.180435 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.180342    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy"
Jul 21 13:15:00.180435 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.180351    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2db7bafd-dca6-417b-9cca-6509def49c0f" containerName="revoke-keys"
Jul 21 13:15:00.184701 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.184682    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd"
Jul 21 13:15:00.191465 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.191447    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-jq8px\""
Jul 21 13:15:00.193651 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.193633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd"
Jul 21 13:15:00.211731 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.211698    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd"]
Jul 21 13:15:00.355680 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.355648    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd"]
Jul 21 13:15:00.359119 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:15:00.359090    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedce33f0_88a0_449b_8784_30b5cabd83dd.slice/crio-c9c8941064f8222694f10d17f557e960a84dcd1a5d305e370be6632e18af1249 WatchSource:0}: Error finding container c9c8941064f8222694f10d17f557e960a84dcd1a5d305e370be6632e18af1249: Status 404 returned error can't find the container with id c9c8941064f8222694f10d17f557e960a84dcd1a5d305e370be6632e18af1249
Jul 21 13:15:00.714724 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.714641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd" event={"ID":"edce33f0-88a0-449b-8784-30b5cabd83dd","Type":"ContainerStarted","Data":"886eaebc3a83a7fda89417db84bb652161bb594b2efebac85e38516cacbff4bf"}
Jul 21 13:15:00.714724 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.714708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd" event={"ID":"edce33f0-88a0-449b-8784-30b5cabd83dd","Type":"ContainerStarted","Data":"c9c8941064f8222694f10d17f557e960a84dcd1a5d305e370be6632e18af1249"}
Jul 21 13:15:00.736982 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:00.735710    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd" podStartSLOduration=0.735691089 podStartE2EDuration="735.691089ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:15:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:15:00.731583281 +0000 UTC m=+2838.516598247" watchObservedRunningTime="2026-07-21 13:15:00.735691089 +0000 UTC m=+2838.520706059"
Jul 21 13:15:01.721326 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:01.721287    2577 generic.go:356] "Generic (PLEG): container finished" podID="edce33f0-88a0-449b-8784-30b5cabd83dd" containerID="886eaebc3a83a7fda89417db84bb652161bb594b2efebac85e38516cacbff4bf" exitCode=0
Jul 21 13:15:01.721713 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:01.721368    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd" event={"ID":"edce33f0-88a0-449b-8784-30b5cabd83dd","Type":"ContainerDied","Data":"886eaebc3a83a7fda89417db84bb652161bb594b2efebac85e38516cacbff4bf"}
Jul 21 13:15:02.859545 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:02.859522    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd"
Jul 21 13:15:03.732379 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:03.732346    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd"
Jul 21 13:15:03.732573 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:03.732388    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29743995-gbvmd" event={"ID":"edce33f0-88a0-449b-8784-30b5cabd83dd","Type":"ContainerDied","Data":"c9c8941064f8222694f10d17f557e960a84dcd1a5d305e370be6632e18af1249"}
Jul 21 13:15:03.732573 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:03.732446    2577 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c9c8941064f8222694f10d17f557e960a84dcd1a5d305e370be6632e18af1249"
Jul 21 13:15:07.843690 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:07.843658    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"]
Jul 21 13:15:07.844164 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:07.843999    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy" containerID="cri-o://b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262" gracePeriod=30
Jul 21 13:15:13.094019 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.093997    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:15:13.151234 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151164    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-certs\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151234 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151199    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sqgnx\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-kube-api-access-sqgnx\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151234 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151217    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-envoy\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151507 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151236    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-socket\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151507 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-token\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151507 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151286    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-data\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151507 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151318    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998141f8-a8cf-4f3c-82f0-5888fa87a053-istiod-ca-cert\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151507 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151357    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-credential-socket\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151507 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151394    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-podinfo\") pod \"998141f8-a8cf-4f3c-82f0-5888fa87a053\" (UID: \"998141f8-a8cf-4f3c-82f0-5888fa87a053\") "
Jul 21 13:15:13.151827 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151498    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:13.151827 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151521    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:13.151827 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151667    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-certs\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.151827 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151686    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-workload-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.151827 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151749    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:13.152125 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.151863    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/998141f8-a8cf-4f3c-82f0-5888fa87a053-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 21 13:15:13.152125 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.152012    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-data" (OuterVolumeSpecName: "istio-data") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:13.153556 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.153529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 21 13:15:13.153673 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.153555    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 21 13:15:13.153734 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.153726    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-kube-api-access-sqgnx" (OuterVolumeSpecName: "kube-api-access-sqgnx") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "kube-api-access-sqgnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:15:13.153796 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.153776    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-token" (OuterVolumeSpecName: "istio-token") pod "998141f8-a8cf-4f3c-82f0-5888fa87a053" (UID: "998141f8-a8cf-4f3c-82f0-5888fa87a053"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 21 13:15:13.252972 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.252921    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sqgnx\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-kube-api-access-sqgnx\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.252972 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.252946    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-envoy\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.252972 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.252977    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-token\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.253203 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.252986    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-data\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.253203 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.252995    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/998141f8-a8cf-4f3c-82f0-5888fa87a053-istiod-ca-cert\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.253203 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.253003    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/998141f8-a8cf-4f3c-82f0-5888fa87a053-credential-socket\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.253203 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.253011    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/998141f8-a8cf-4f3c-82f0-5888fa87a053-istio-podinfo\") on node \"ip-10-0-143-161.ec2.internal\" DevicePath \"\""
Jul 21 13:15:13.781022 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.780990    2577 generic.go:356] "Generic (PLEG): container finished" podID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerID="b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262" exitCode=0
Jul 21 13:15:13.781186 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.781035    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" event={"ID":"998141f8-a8cf-4f3c-82f0-5888fa87a053","Type":"ContainerDied","Data":"b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262"}
Jul 21 13:15:13.781186 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.781078    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97" event={"ID":"998141f8-a8cf-4f3c-82f0-5888fa87a053","Type":"ContainerDied","Data":"2d8b7715bd7feedb44b2a64f4bf013bdb85827da0ad561a59ab9385937ccb60f"}
Jul 21 13:15:13.781186 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.781081    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"
Jul 21 13:15:13.781186 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.781099    2577 scope.go:117] "RemoveContainer" containerID="b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262"
Jul 21 13:15:13.790705 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.790689    2577 scope.go:117] "RemoveContainer" containerID="95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f"
Jul 21 13:15:13.797945 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.797927    2577 scope.go:117] "RemoveContainer" containerID="b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262"
Jul 21 13:15:13.798204 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:15:13.798185    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262\": container with ID starting with b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262 not found: ID does not exist" containerID="b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262"
Jul 21 13:15:13.798253 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.798213    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262"} err="failed to get container status \"b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262\": rpc error: code = NotFound desc = could not find container \"b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262\": container with ID starting with b45e69b31cc5f9aabe62d0d3dd9a3c3376e5ffb3d81e2705c3cfe3fecd3a0262 not found: ID does not exist"
Jul 21 13:15:13.798253 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.798229    2577 scope.go:117] "RemoveContainer" containerID="95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f"
Jul 21 13:15:13.798462 ip-10-0-143-161 kubenswrapper[2577]: E0721 13:15:13.798444    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f\": container with ID starting with 95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f not found: ID does not exist" containerID="95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f"
Jul 21 13:15:13.798515 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.798472    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f"} err="failed to get container status \"95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f\": rpc error: code = NotFound desc = could not find container \"95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f\": container with ID starting with 95bda69acdf7d91fa8f2546ece76aba7f3ce25662dcf7122506602f1b986279f not found: ID does not exist"
Jul 21 13:15:13.803186 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.803167    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"]
Jul 21 13:15:13.806428 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:13.806407    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-b1b184-openshift-default-5d5d4d98cf-mcv97"]
Jul 21 13:15:14.258730 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:14.258704    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f29z8_8e044a3a-dd4c-44fd-bf33-feb5494e5dfe/manager/0.log"
Jul 21 13:15:14.382521 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:14.382493    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zw2vt_3d472edf-63c8-4c08-a6a0-99b82436388d/manager/0.log"
Jul 21 13:15:14.677822 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:14.677739    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c5gbn_2782bae9-cb6a-435b-b17c-f620c3b7a70d/server/0.log"
Jul 21 13:15:14.811883 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:14.811853    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xb84h_cadc5390-e3f0-4511-9d15-12d6a9bb3da3/manager/0.log"
Jul 21 13:15:14.832425 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:14.832401    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" path="/var/lib/kubelet/pods/998141f8-a8cf-4f3c-82f0-5888fa87a053/volumes"
Jul 21 13:15:15.222472 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:15.222447    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-zbxvm_fd460f8b-5eee-498e-b818-5c78666719dc/copy-manifests/0.log"
Jul 21 13:15:15.229569 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:15.229548    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-zbxvm_fd460f8b-5eee-498e-b818-5c78666719dc/manager/0.log"
Jul 21 13:15:15.868714 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:15.868680    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743980-tcv94_f0303ce0-8657-4e8d-9ac7-1f40ca66df53/cleanup/0.log"
Jul 21 13:15:15.990972 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:15.990929    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29743995-gbvmd_edce33f0-88a0-449b-8784-30b5cabd83dd/cleanup/0.log"
Jul 21 13:15:17.642920 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:17.642891    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x8gz5_ee0753d5-545e-4843-ad66-cffa4cc0d3ca/manager/0.log"
Jul 21 13:15:17.876385 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:17.876351    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-t2dqs_d956aff0-9f68-49d3-be2e-58ee02f99848/kuadrant-console-plugin/0.log"
Jul 21 13:15:17.996573 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:17.996548    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-phwn2_2e701718-8ca7-4f2a-9914-1528c85cee0a/registry-server/0.log"
Jul 21 13:15:18.125905 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:18.125877    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-65lgr_01f4cb0c-b4c6-48e2-bcd0-f899becae7b2/manager/0.log"
Jul 21 13:15:18.391971 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:18.391889    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qnpdj_a730b152-a7cf-47c2-a015-5d60630ed037/manager/0.log"
Jul 21 13:15:18.764094 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:18.764068    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678g2wxn_c28b5daa-f9df-41d9-b524-dea8642cd8d1/istio-proxy/0.log"
Jul 21 13:15:19.148157 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:19.148087    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-z6bnl_e8be54e4-130a-41d2-8b31-d27d3d1754c9/kube-auth-proxy/0.log"
Jul 21 13:15:19.986983 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:19.986930    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-7xjbm_426020a0-6085-4d58-8f72-506c0a9a2876/main/0.log"
Jul 21 13:15:20.233662 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:20.233636    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wblcn_d227d846-f642-4883-92df-da804f6035dd/main/0.log"
Jul 21 13:15:26.682578 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.682539    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lpgz9/must-gather-cd889"]
Jul 21 13:15:26.683067 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683040    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy"
Jul 21 13:15:26.683067 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683060    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy"
Jul 21 13:15:26.683192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683085    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy"
Jul 21 13:15:26.683192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683095    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy"
Jul 21 13:15:26.683192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683121    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edce33f0-88a0-449b-8784-30b5cabd83dd" containerName="cleanup"
Jul 21 13:15:26.683192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683130    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="edce33f0-88a0-449b-8784-30b5cabd83dd" containerName="cleanup"
Jul 21 13:15:26.683192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683144    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy"
Jul 21 13:15:26.683192 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683152    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b66cccae-c3e7-4e6f-a150-200ad5c5b8e7" containerName="istio-proxy"
Jul 21 13:15:26.683490 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683230    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy"
Jul 21 13:15:26.683490 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683242    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="edce33f0-88a0-449b-8784-30b5cabd83dd" containerName="cleanup"
Jul 21 13:15:26.683490 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.683253    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="998141f8-a8cf-4f3c-82f0-5888fa87a053" containerName="istio-proxy"
Jul 21 13:15:26.686392 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.686371    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:26.688935 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.688917    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpgz9\"/\"kube-root-ca.crt\""
Jul 21 13:15:26.689795 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.689775    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-lpgz9\"/\"openshift-service-ca.crt\""
Jul 21 13:15:26.689917 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.689779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-lpgz9\"/\"default-dockercfg-4r69j\""
Jul 21 13:15:26.695186 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.695161    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpgz9/must-gather-cd889"]
Jul 21 13:15:26.761193 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.761165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqdd4\" (UniqueName: \"kubernetes.io/projected/6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc-kube-api-access-fqdd4\") pod \"must-gather-cd889\" (UID: \"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc\") " pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:26.761324 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.761213    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc-must-gather-output\") pod \"must-gather-cd889\" (UID: \"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc\") " pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:26.862338 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.862306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqdd4\" (UniqueName: \"kubernetes.io/projected/6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc-kube-api-access-fqdd4\") pod \"must-gather-cd889\" (UID: \"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc\") " pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:26.862508 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.862360    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc-must-gather-output\") pod \"must-gather-cd889\" (UID: \"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc\") " pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:26.862677 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.862660    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc-must-gather-output\") pod \"must-gather-cd889\" (UID: \"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc\") " pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:26.874154 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.874129    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqdd4\" (UniqueName: \"kubernetes.io/projected/6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc-kube-api-access-fqdd4\") pod \"must-gather-cd889\" (UID: \"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc\") " pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:26.996346 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:26.996317    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpgz9/must-gather-cd889"
Jul 21 13:15:27.137791 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:27.137763    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpgz9/must-gather-cd889"]
Jul 21 13:15:27.139802 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:15:27.139773    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6cf6a2ed_46e9_4dc2_9c87_3b96c12139dc.slice/crio-29bac89782d998588a27f4598ba8ff1528eab78d481a49b0c459b1261b44c030 WatchSource:0}: Error finding container 29bac89782d998588a27f4598ba8ff1528eab78d481a49b0c459b1261b44c030: Status 404 returned error can't find the container with id 29bac89782d998588a27f4598ba8ff1528eab78d481a49b0c459b1261b44c030
Jul 21 13:15:27.845406 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:27.845374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpgz9/must-gather-cd889" event={"ID":"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc","Type":"ContainerStarted","Data":"29bac89782d998588a27f4598ba8ff1528eab78d481a49b0c459b1261b44c030"}
Jul 21 13:15:28.853084 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:28.853046    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpgz9/must-gather-cd889" event={"ID":"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc","Type":"ContainerStarted","Data":"475978aea1c798e6dd26b092efa59e926309737868d14fd4c0d56242b1258bd2"}
Jul 21 13:15:28.853627 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:28.853594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpgz9/must-gather-cd889" event={"ID":"6cf6a2ed-46e9-4dc2-9c87-3b96c12139dc","Type":"ContainerStarted","Data":"112ceb86dd354c7524b3d6ddb542fdac7e11f57480aae63db9a2663b71a0cf30"}
Jul 21 13:15:28.871467 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:28.871408    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lpgz9/must-gather-cd889" podStartSLOduration=1.8708549140000001 podStartE2EDuration="2.871387094s" totalImagesPullingTime="1.00053218s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 13:15:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:15:28.868811254 +0000 UTC m=+2866.653826213" watchObservedRunningTime="2026-07-21 13:15:28.871387094 +0000 UTC m=+2866.656402062"
Jul 21 13:15:29.757107 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:29.757079    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-h9wst_4b0a4fdc-9ec4-4894-96f7-2e68c62f4bc6/global-pull-secret-syncer/0.log"
Jul 21 13:15:29.972549 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:29.972512    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-z5xzk_157a0b56-df6f-4a64-800a-84d8a7596eb9/konnectivity-agent/0.log"
Jul 21 13:15:30.051497 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:30.051425    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-143-161.ec2.internal_4823430b3b83d7b0cb59efe2028dc586/haproxy/0.log"
Jul 21 13:15:34.898804 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:34.898716    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-x8gz5_ee0753d5-545e-4843-ad66-cffa4cc0d3ca/manager/0.log"
Jul 21 13:15:35.032265 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:35.032237    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-t2dqs_d956aff0-9f68-49d3-be2e-58ee02f99848/kuadrant-console-plugin/0.log"
Jul 21 13:15:35.095399 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:35.095358    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-phwn2_2e701718-8ca7-4f2a-9914-1528c85cee0a/registry-server/0.log"
Jul 21 13:15:35.281905 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:35.281876    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-65lgr_01f4cb0c-b4c6-48e2-bcd0-f899becae7b2/manager/0.log"
Jul 21 13:15:35.381514 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:35.381474    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-qnpdj_a730b152-a7cf-47c2-a015-5d60630ed037/manager/0.log"
Jul 21 13:15:37.444466 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:37.444427    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8j7h4_08f3b63c-95ca-45c5-81e9-b4d7c12e1451/node-exporter/0.log"
Jul 21 13:15:37.471021 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:37.470869    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8j7h4_08f3b63c-95ca-45c5-81e9-b4d7c12e1451/kube-rbac-proxy/0.log"
Jul 21 13:15:37.496762 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:37.496666    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8j7h4_08f3b63c-95ca-45c5-81e9-b4d7c12e1451/init-textfile/0.log"
Jul 21 13:15:39.411865 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.411830    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"]
Jul 21 13:15:39.417530 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.417498    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.424323 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.424297    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"]
Jul 21 13:15:39.593744 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.593705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-podres\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.593941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.593774    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnp2q\" (UniqueName: \"kubernetes.io/projected/b84b35d6-1be7-4f5c-923b-f04f0e948aff-kube-api-access-lnp2q\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.593941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.593819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-proc\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.593941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.593848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-lib-modules\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.593941 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.593892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-sys\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694387 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694287    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-proc\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694387 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694331    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-lib-modules\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694621 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694400    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-sys\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694621 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-podres\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694621 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694495    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-sys\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694621 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-lib-modules\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694621 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694502    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnp2q\" (UniqueName: \"kubernetes.io/projected/b84b35d6-1be7-4f5c-923b-f04f0e948aff-kube-api-access-lnp2q\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694621 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-proc\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.694621 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.694608    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b84b35d6-1be7-4f5c-923b-f04f0e948aff-podres\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.705713 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.705684    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnp2q\" (UniqueName: \"kubernetes.io/projected/b84b35d6-1be7-4f5c-923b-f04f0e948aff-kube-api-access-lnp2q\") pod \"perf-node-gather-daemonset-ffq4c\" (UID: \"b84b35d6-1be7-4f5c-923b-f04f0e948aff\") " pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.730682 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.730643    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:39.844498 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.844451    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-rg7w2_e807dafc-b707-40f5-aaf7-e125c273bf7b/networking-console-plugin/0.log"
Jul 21 13:15:39.913989 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.913939    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"]
Jul 21 13:15:39.918219 ip-10-0-143-161 kubenswrapper[2577]: W0721 13:15:39.918172    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb84b35d6_1be7_4f5c_923b_f04f0e948aff.slice/crio-48549e7012b0d56647e6858151ff4af7d2026f0a1ec31ecf6d226168e3d990b1 WatchSource:0}: Error finding container 48549e7012b0d56647e6858151ff4af7d2026f0a1ec31ecf6d226168e3d990b1: Status 404 returned error can't find the container with id 48549e7012b0d56647e6858151ff4af7d2026f0a1ec31ecf6d226168e3d990b1
Jul 21 13:15:39.935927 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:39.935884    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c" event={"ID":"b84b35d6-1be7-4f5c-923b-f04f0e948aff","Type":"ContainerStarted","Data":"48549e7012b0d56647e6858151ff4af7d2026f0a1ec31ecf6d226168e3d990b1"}
Jul 21 13:15:40.943129 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:40.943094    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c" event={"ID":"b84b35d6-1be7-4f5c-923b-f04f0e948aff","Type":"ContainerStarted","Data":"329bca74fe4ffb20c1b428bff6dbb22ab8bdcc0e89756abce59deeba44b96e0d"}
Jul 21 13:15:40.943593 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:40.943202    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:40.960981 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:40.960913    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c" podStartSLOduration=1.960893231 podStartE2EDuration="1.960893231s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 13:15:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 13:15:40.959446781 +0000 UTC m=+2878.744461768" watchObservedRunningTime="2026-07-21 13:15:40.960893231 +0000 UTC m=+2878.745908198"
Jul 21 13:15:42.616352 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:42.616318    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvnqn_02461782-afbd-4b5a-b082-09179a2f4471/dns/0.log"
Jul 21 13:15:42.638120 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:42.638090    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-bvnqn_02461782-afbd-4b5a-b082-09179a2f4471/kube-rbac-proxy/0.log"
Jul 21 13:15:42.760562 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:42.760527    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-76zlc_5f1a2f32-365c-48b2-b9a8-56a5da0c42f0/dns-node-resolver/0.log"
Jul 21 13:15:43.434355 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:43.434283    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-zsqm9_9f6ab4a5-3a99-439c-934c-e07efea84081/node-ca/0.log"
Jul 21 13:15:44.351978 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:44.351933    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678g2wxn_c28b5daa-f9df-41d9-b524-dea8642cd8d1/istio-proxy/0.log"
Jul 21 13:15:44.678138 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:44.678058    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-z6bnl_e8be54e4-130a-41d2-8b31-d27d3d1754c9/kube-auth-proxy/0.log"
Jul 21 13:15:45.579188 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:45.579163    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dwpsm_a8cb2539-aa9d-44c0-b732-669e6f748287/serve-healthcheck-canary/0.log"
Jul 21 13:15:46.264865 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:46.264840    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqr6l_5c158b7a-8e3b-483d-995f-59ce8614629c/kube-rbac-proxy/0.log"
Jul 21 13:15:46.289244 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:46.289219    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqr6l_5c158b7a-8e3b-483d-995f-59ce8614629c/exporter/0.log"
Jul 21 13:15:46.315242 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:46.315216    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-bqr6l_5c158b7a-8e3b-483d-995f-59ce8614629c/extractor/0.log"
Jul 21 13:15:46.959350 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:46.959321    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-lpgz9/perf-node-gather-daemonset-ffq4c"
Jul 21 13:15:48.697279 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:48.697250    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-f29z8_8e044a3a-dd4c-44fd-bf33-feb5494e5dfe/manager/0.log"
Jul 21 13:15:48.954791 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:48.954704    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-zw2vt_3d472edf-63c8-4c08-a6a0-99b82436388d/manager/0.log"
Jul 21 13:15:49.163073 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:49.163035    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-c5gbn_2782bae9-cb6a-435b-b17c-f620c3b7a70d/server/0.log"
Jul 21 13:15:49.254244 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:49.254214    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-xb84h_cadc5390-e3f0-4511-9d15-12d6a9bb3da3/manager/0.log"
Jul 21 13:15:49.446905 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:49.446877    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-zbxvm_fd460f8b-5eee-498e-b818-5c78666719dc/manager/0.log"
Jul 21 13:15:49.474865 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:49.474771    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb6668c8b-zbxvm_fd460f8b-5eee-498e-b818-5c78666719dc/copy-manifests/0.log"
Jul 21 13:15:49.875612 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:49.875582    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678g2wxn_c28b5daa-f9df-41d9-b524-dea8642cd8d1/istio-proxy/0.log"
Jul 21 13:15:50.131665 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:50.131580    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-z6bnl_e8be54e4-130a-41d2-8b31-d27d3d1754c9/kube-auth-proxy/0.log"
Jul 21 13:15:50.729610 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:50.729581    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-rscbm_0e22ed42-bf05-432a-b045-3506cb8b144c/openshift-lws-operator/0.log"
Jul 21 13:15:51.483387 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:51.483358    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678g2wxn_c28b5daa-f9df-41d9-b524-dea8642cd8d1/istio-proxy/0.log"
Jul 21 13:15:51.732129 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:51.732107    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b686cc986-z6bnl_e8be54e4-130a-41d2-8b31-d27d3d1754c9/kube-auth-proxy/0.log"
Jul 21 13:15:55.512804 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:55.512781    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-nh25g_2cc15a8d-64f3-49bd-a9f2-74a8f210f260/migrator/0.log"
Jul 21 13:15:55.537403 ip-10-0-143-161 kubenswrapper[2577]: I0721 13:15:55.537373    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-nh25g_2cc15a8d-64f3-49bd-a9f2-74a8f210f260/graceful-termination/0.log"