Jul 22 12:34:59.678055 ip-10-0-128-197 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 22 12:34:59.678066 ip-10-0-128-197 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 22 12:34:59.678074 ip-10-0-128-197 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 22 12:34:59.678361 ip-10-0-128-197 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 22 12:35:10.463965 ip-10-0-128-197 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 22 12:35:10.463979 ip-10-0-128-197 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot b9b5e828b76342ab991e0809ea811bec --
Jul 22 12:36:55.417041 ip-10-0-128-197 systemd[1]: Starting Kubernetes Kubelet...
Jul 22 12:36:55.813200 ip-10-0-128-197 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:36:55.813200 ip-10-0-128-197 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 22 12:36:55.813200 ip-10-0-128-197 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:36:55.813200 ip-10-0-128-197 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 22 12:36:55.813200 ip-10-0-128-197 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 22 12:36:55.814996 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.814911    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 22 12:36:55.823048 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823025    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:36:55.823048 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823046    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:36:55.823048 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823050    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:36:55.823048 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823054    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:36:55.823048 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823057    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823060    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823063    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823066    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823069    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823072    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823075    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823078    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823080    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823083    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823085    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823088    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823090    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823093    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823095    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823098    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823100    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823103    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823105    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823108    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:36:55.823241 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823110    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823112    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823115    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823117    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823119    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823122    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823124    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823127    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823129    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823132    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823135    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823137    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823140    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823142    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823145    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823148    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823150    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823152    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823155    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823157    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:36:55.823713 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823160    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823163    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823165    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823167    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823171    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823173    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823176    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823178    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823180    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823182    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823185    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823188    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823190    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823193    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823195    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823198    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823200    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823203    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823205    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:36:55.824222 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823207    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823210    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823214    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823219    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823222    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823225    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823227    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823230    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823233    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823235    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823238    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823241    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823244    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823247    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823249    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823252    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823254    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823257    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823261    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823264    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:36:55.824673 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823268    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823271    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823274    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823276    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823279    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823281    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823284    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823286    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823288    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823291    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823293    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823296    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823298    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823301    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823303    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.823306    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824054    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824063    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824066    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824071    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:36:55.825181 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824073    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824076    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824080    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824082    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824085    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824087    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824090    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824093    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824096    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824098    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824101    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824103    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824106    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824108    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824111    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824113    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824116    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824118    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824121    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824123    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:36:55.825646 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824126    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824128    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824130    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824133    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824135    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824137    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824139    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824142    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824146    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824150    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824155    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824158    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824161    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824164    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824167    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824169    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824172    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824174    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824177    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:36:55.826268 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824179    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824181    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824184    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824186    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824189    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824192    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824195    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824199    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824202    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824204    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824207    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824210    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824212    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824215    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824217    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824219    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824222    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824224    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824227    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:36:55.826734 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824231    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824234    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824236    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824240    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824243    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824246    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824248    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824251    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824253    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824256    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824258    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824261    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824264    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824266    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824269    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824271    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824273    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824276    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824278    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824280    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:36:55.827218 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824283    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824286    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824288    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824291    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824293    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824296    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824298    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824300    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824303    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824306    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824308    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824311    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824313    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824316    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824318    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824321    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824323    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824692    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824697    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824699    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:36:55.827693 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824702    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824706    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824710    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824713    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824715    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824718    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824721    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824724    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824727    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824730    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824732    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824735    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824738    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824742    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824766    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824769    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824772    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824775    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824778    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:36:55.828192 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824781    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824784    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824787    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824789    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824792    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824794    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824797    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824799    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824802    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824805    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824808    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824810    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824813    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824815    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824820    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824824    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824827    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824831    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824835    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824839    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:36:55.828644 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824842    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824845    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824847    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824850    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824852    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824854    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824857    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824861    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824864    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824866    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824868    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824871    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824873    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824876    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824878    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824881    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824884    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824886    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824889    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824891    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:36:55.829145 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824894    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824897    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824903    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824906    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824909    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824912    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824915    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824917    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824919    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824924    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824926    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824928    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824931    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824933    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824936    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824938    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824941    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824943    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824946    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824948    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:36:55.829639 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824951    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824953    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824957    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824960    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824963    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824966    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824969    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824972    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824974    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824977    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824979    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824982    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824985    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824987    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824991    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824993    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.824995    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825066    2563 flags.go:64] FLAG: --address="0.0.0.0"
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825073    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825081    2563 flags.go:64] FLAG: --anonymous-auth="true"
Jul 22 12:36:55.830136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825085    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825089    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825092    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825096    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825100    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825103    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825106    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825109    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825112    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825115    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825118    2563 flags.go:64] FLAG: --cgroup-root=""
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825121    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825123    2563 flags.go:64] FLAG: --client-ca-file=""
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825127    2563 flags.go:64] FLAG: --cloud-provider="external"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825130    2563 flags.go:64] FLAG: --cluster-dns="[]"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825134    2563 flags.go:64] FLAG: --cluster-domain=""
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825137    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825140    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825143    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825146    2563 flags.go:64] FLAG: --container-log-max-files="5"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825149    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825153    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825156    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 22 12:36:55.830646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825159    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825162    2563 flags.go:64] FLAG: --contention-profiling="false"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825165    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825169    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825172    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825175    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825179    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825182    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825185    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825188    2563 flags.go:64] FLAG: --enable-load-reader="false"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825191    2563 flags.go:64] FLAG: --enable-server="true"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825194    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825201    2563 flags.go:64] FLAG: --event-burst="100"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825205    2563 flags.go:64] FLAG: --event-qps="50"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825208    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825210    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825213    2563 flags.go:64] FLAG: --eviction-hard=""
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825220    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825223    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825227    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825229    2563 flags.go:64] FLAG: --eviction-soft=""
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825232    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825235    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825238    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825241    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 22 12:36:55.831211 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825244    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825247    2563 flags.go:64] FLAG: --fail-swap-on="true"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825249    2563 flags.go:64] FLAG: --feature-gates=""
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825254    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825257    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825260    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825263    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825266    2563 flags.go:64] FLAG: --healthz-port="10248"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825269    2563 flags.go:64] FLAG: --help="false"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825272    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-128-197.ec2.internal"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825276    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825279    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825282    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825285    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825288    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825291    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825294    2563 flags.go:64] FLAG: --image-service-endpoint=""
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825297    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825300    2563 flags.go:64] FLAG: --kube-api-burst="100"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825302    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825306    2563 flags.go:64] FLAG: --kube-api-qps="50"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825308    2563 flags.go:64] FLAG: --kube-reserved=""
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825311    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825314    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 22 12:36:55.831882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825316    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825319    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825322    2563 flags.go:64] FLAG: --lock-file=""
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825324    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825327    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825330    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825335    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825338    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825340    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825343    2563 flags.go:64] FLAG: --logging-format="text"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825346    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825352    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825355    2563 flags.go:64] FLAG: --manifest-url=""
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825357    2563 flags.go:64] FLAG: --manifest-url-header=""
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825361    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825364    2563 flags.go:64] FLAG: --max-open-files="1000000"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825368    2563 flags.go:64] FLAG: --max-pods="110"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825371    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825375    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825377    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825380    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825383    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825386    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825388    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825396    2563 flags.go:64] FLAG: --node-status-max-images="50"
Jul 22 12:36:55.832459 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825399    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825402    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825404    2563 flags.go:64] FLAG: --pod-cidr=""
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825407    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825413    2563 flags.go:64] FLAG: --pod-manifest-path=""
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825416    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825419    2563 flags.go:64] FLAG: --pods-per-core="0"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825422    2563 flags.go:64] FLAG: --port="10250"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825425    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825428    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-028464ad256fdeb1d"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825431    2563 flags.go:64] FLAG: --qos-reserved=""
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825434    2563 flags.go:64] FLAG: --read-only-port="10255"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825437    2563 flags.go:64] FLAG: --register-node="true"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825439    2563 flags.go:64] FLAG: --register-with-taints=""
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825447    2563 flags.go:64] FLAG: --registry-burst="10"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825450    2563 flags.go:64] FLAG: --registry-qps="5"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825452    2563 flags.go:64] FLAG: --reserved-cpus=""
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825455    2563 flags.go:64] FLAG: --reserved-memory=""
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825459    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825463    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825466    2563 flags.go:64] FLAG: --rotate-certificates="false"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825468    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825471    2563 flags.go:64] FLAG: --runonce="false"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825474    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825477    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 22 12:36:55.833079 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825481    2563 flags.go:64] FLAG: --seccomp-default="false"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825483    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825486    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825489    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825492    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825495    2563 flags.go:64] FLAG: --storage-driver-password="root"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825498    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825500    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825503    2563 flags.go:64] FLAG: --storage-driver-user="root"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825506    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825509    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825511    2563 flags.go:64] FLAG: --system-cgroups=""
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825514    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825519    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825522    2563 flags.go:64] FLAG: --tls-cert-file=""
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825524    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825528    2563 flags.go:64] FLAG: --tls-min-version=""
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825531    2563 flags.go:64] FLAG: --tls-private-key-file=""
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825534    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825537    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825539    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825542    2563 flags.go:64] FLAG: --v="2"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825546    2563 flags.go:64] FLAG: --version="false"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825550    2563 flags.go:64] FLAG: --vmodule=""
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825553    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 22 12:36:55.833686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.825556    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825684    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825688    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825691    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825694    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825697    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825699    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825704    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825707    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825709    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825712    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825714    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825717    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825720    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825722    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825724    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825727    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825729    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825732    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:36:55.834340 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825734    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825737    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825739    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825741    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825761    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825764    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825767    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825769    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825772    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825774    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825777    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825779    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825783    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825786    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825790    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825792    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825795    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825797    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825800    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825802    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:36:55.834812 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825807    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825809    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825812    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825815    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825817    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825820    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825822    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825825    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825827    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825830    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825832    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825835    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825837    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825840    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825842    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825844    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825847    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825849    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825851    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825854    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:36:55.835296 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825856    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825858    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825861    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825863    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825866    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825869    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825874    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825876    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825879    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825881    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825884    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825886    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825890    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825892    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825895    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825897    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825900    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825902    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825905    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:36:55.835805 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825907    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825911    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825915    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825918    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825921    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825924    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825927    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825929    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825932    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825935    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825937    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825940    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825942    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825945    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825947    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825950    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825952    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825955    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825957    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:36:55.836274 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825961    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825964    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.825967    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.826612    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.833260    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.833275    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.834069    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.834077    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834133    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834139    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834142    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834146    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834148    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834151    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:36:55.836727 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834154    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834157    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834159    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834162    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834164    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834167    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834169    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834172    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834174    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834177    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834179    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834181    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834185    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834189    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834191    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834194    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834197    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834199    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834202    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:36:55.837205 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834204    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834206    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834209    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834212    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834215    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834218    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834221    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834223    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834226    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834228    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834231    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834233    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834235    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834238    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834240    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834244    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834248    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834251    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834254    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834256    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:36:55.837661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834259    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834261    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834264    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834266    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834268    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834271    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834273    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834275    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834278    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834280    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834282    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834285    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834288    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834290    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834293    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834295    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834299    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834302    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834304    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834307    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:36:55.838148 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834309    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834312    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834315    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834317    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834320    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834322    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834326    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834329    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834332    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834335    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834337    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834341    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834344    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834346    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834349    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834352    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834355    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834357    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834360    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:36:55.839098 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834363    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834366    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834368    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834370    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834373    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834376    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834378    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834381    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834383    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834386    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834389    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834392    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834394    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834397    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834399    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:36:55.839585 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.834405    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834518    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834522    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834525    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834528    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834531    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834533    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834536    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834538    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834541    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834543    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834546    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834548    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834551    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834553    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834556    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834559    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834561    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834564    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834566    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 22 12:36:55.839987 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834569    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834571    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834573    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834576    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834578    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834580    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834583    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834586    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834588    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834591    2563 feature_gate.go:328] unrecognized feature gate: Example2
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834593    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834596    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834598    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834601    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834603    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834606    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834608    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834611    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834613    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834615    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 22 12:36:55.840543 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834618    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834620    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834622    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834625    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834627    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834631    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834633    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834637    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834639    2563 feature_gate.go:328] unrecognized feature gate: Example
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834642    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834644    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834647    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834649    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834651    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834654    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834656    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834658    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834661    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834663    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834666    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 22 12:36:55.841174 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834668    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834671    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834674    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834676    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834678    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834681    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834683    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834686    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834688    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834691    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834693    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834695    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834698    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834701    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834705    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834708    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834710    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834713    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834715    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 22 12:36:55.842043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834718    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834720    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834722    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834725    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834727    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834730    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834732    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834734    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834736    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834739    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834741    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834759    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834763    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834766    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834768    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834771    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834774    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834777    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834779    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834781    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 22 12:36:55.842683 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:55.834785    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.834791    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.834898    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.836799    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.838158    2563 server.go:1019] "Starting client certificate rotation"
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.838276    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.838328    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.840512    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:36:55.843207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.842024    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 22 12:36:55.851539 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.851520    2563 log.go:25] "Validated CRI v1 runtime API"
Jul 22 12:36:55.854002 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.853987    2563 log.go:25] "Validated CRI v1 image API"
Jul 22 12:36:55.854809 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.854795    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 22 12:36:55.859924 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.859904    2563 fs.go:135] Filesystem UUIDs: map[4a358bca-a071-430a-859e-f539c05359f1:/dev/nvme0n1p4 6dec301e-ddc2-4d28-b6e1-f371e974e25d:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 22 12:36:55.859974 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.859924    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 22 12:36:55.865342 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.865240    2563 manager.go:216] Machine: {Timestamp:2026-07-22 12:36:55.863671484 +0000 UTC m=+0.328569524 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3085141 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29e085757c0a9fe7c2b5004f887cb0 SystemUUID:ec29e085-757c-0a9f-e7c2-b5004f887cb0 BootID:b9b5e828-b763-42ab-991e-0809ea811bec Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:52:9d:7e:3f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:52:9d:7e:3f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:f2:e3:34:3f:b6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 22 12:36:55.865342 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.865337    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 22 12:36:55.865449 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.865411    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 22 12:36:55.868968 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.868938    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 22 12:36:55.869119 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.868972    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-197.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 22 12:36:55.870442 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.870431    2563 topology_manager.go:138] "Creating topology manager with none policy"
Jul 22 12:36:55.870481 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.870444    2563 container_manager_linux.go:309] "Creating device plugin manager"
Jul 22 12:36:55.870481 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.870452    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:36:55.871254 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.871244    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 22 12:36:55.872499 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.872489    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 22 12:36:55.875800 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.875788    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:36:55.875930 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.875920    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 22 12:36:55.876625 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.876615    2563 kubelet.go:486] "Attempting to sync node with API server"
Jul 22 12:36:55.876657 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.876634    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 22 12:36:55.876657 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.876648    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 22 12:36:55.876725 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.876658    2563 kubelet.go:398] "Adding apiserver pod source"
Jul 22 12:36:55.876725 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.876666    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 22 12:36:55.877061 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.877050    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:36:55.877094 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.877069    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 22 12:36:55.884466 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.884449    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 22 12:36:55.886405 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.886388    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 22 12:36:55.886482 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.886427    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 22 12:36:55.887691 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887677    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887697    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887705    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887712    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887724    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887732    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887760    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887769    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 22 12:36:55.887776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887779    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 22 12:36:55.888050 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887787    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 22 12:36:55.888050 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887800    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 22 12:36:55.888050 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.887809    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 22 12:36:55.888618 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.888606    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 22 12:36:55.888667 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.888620    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 22 12:36:55.889815 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.889794    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:36:55.891922 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.891907    2563 server.go:1268] "Started kubelet"
Jul 22 12:36:55.892004 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.891968    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 22 12:36:55.892137 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.892004    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 22 12:36:55.892292 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.892273    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 22 12:36:55.896133 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.896106    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-197.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 22 12:36:55.896215 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.896199    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-197.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 22 12:36:55.896363 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.896342    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 22 12:36:55.896518 ip-10-0-128-197 systemd[1]: Started Kubernetes Kubelet.
Jul 22 12:36:55.899548 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.899083    2563 server.go:310] "Adding debug handlers to kubelet server"
Jul 22 12:36:55.899893 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.899720    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 22 12:36:55.900831 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.900813    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 22 12:36:55.905874 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.905854    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 22 12:36:55.905960 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.905867    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 22 12:36:55.905960 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.905927    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:55.906521 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.906505    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 22 12:36:55.906607 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.906510    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 22 12:36:55.906607 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.906543    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 22 12:36:55.906702 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.906628    2563 reconstruct.go:97] "Volume reconstruction finished"
Jul 22 12:36:55.906702 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.906639    2563 reconciler.go:29] "Reconciler: start to sync state"
Jul 22 12:36:55.906818 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.905618    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18c49cdacc6a4f74  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-07-22 12:36:55.891881844 +0000 UTC m=+0.356779886,LastTimestamp:2026-07-22 12:36:55.891881844 +0000 UTC m=+0.356779886,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}"
Jul 22 12:36:55.906906 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.906890    2563 factory.go:55] Registering systemd factory
Jul 22 12:36:55.906957 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.906909    2563 factory.go:223] Registration of the systemd container factory successfully
Jul 22 12:36:55.907144 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.907132    2563 factory.go:153] Registering CRI-O factory
Jul 22 12:36:55.907189 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.907146    2563 factory.go:223] Registration of the crio container factory successfully
Jul 22 12:36:55.907189 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.907186    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 22 12:36:55.907264 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.907227    2563 factory.go:103] Registering Raw factory
Jul 22 12:36:55.907264 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.907239    2563 manager.go:1196] Started watching for new ooms in manager
Jul 22 12:36:55.908039 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.908021    2563 manager.go:349] Starting recovery of all containers
Jul 22 12:36:55.915874 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.915832    2563 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-197.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 22 12:36:55.915953 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.915901    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 22 12:36:55.921384 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.921213    2563 manager.go:354] Recovery completed
Jul 22 12:36:55.925862 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.925851    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:55.928852 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.928830    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:55.928911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.928868    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:55.928911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.928884    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:55.929538 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.929524    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 22 12:36:55.929538 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.929538    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 22 12:36:55.929636 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.929556    2563 state_mem.go:36] "Initialized new in-memory state store"
Jul 22 12:36:55.929636 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.929585    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:55.930364 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.930349    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:55.930428 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.930377    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:55.930428 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.930397    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:55.930950 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.930935    2563 policy_none.go:49] "None policy: Start"
Jul 22 12:36:55.930982 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.930961    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 22 12:36:55.930982 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.930976    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 22 12:36:55.931260 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.931189    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18c49cdace9e6e1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-07-22 12:36:55.928851997 +0000 UTC m=+0.393750039,LastTimestamp:2026-07-22 12:36:55.928851997 +0000 UTC m=+0.393750039,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}"
Jul 22 12:36:55.932224 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.932211    2563 policy_none.go:47] "Start"
Jul 22 12:36:55.953596 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.953528    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18c49cdace9eca63  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-07-22 12:36:55.928875619 +0000 UTC m=+0.393773665,LastTimestamp:2026-07-22 12:36:55.928875619 +0000 UTC m=+0.393773665,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}"
Jul 22 12:36:55.963925 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.963909    2563 manager.go:337] "Starting Device Plugin manager"
Jul 22 12:36:55.964013 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.963947    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 22 12:36:55.964013 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.963963    2563 server.go:87] "Starting device plugin registration server"
Jul 22 12:36:55.964220 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.964204    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 22 12:36:55.964304 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.964221    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 22 12:36:55.964346 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.964299    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 22 12:36:55.964399 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.964377    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 22 12:36:55.964399 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.964386    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 22 12:36:55.964953 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.964931    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 22 12:36:55.965034 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.964980    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:55.970796 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.970703    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18c49cdace9efdc2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-07-22 12:36:55.92888877 +0000 UTC m=+0.393786809,LastTimestamp:2026-07-22 12:36:55.92888877 +0000 UTC m=+0.393786809,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}"
Jul 22 12:36:55.982277 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.982203    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-197.ec2.internal.18c49cdace9e6e1d\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18c49cdace9e6e1d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-07-22 12:36:55.928851997 +0000 UTC m=+0.393750039,LastTimestamp:2026-07-22 12:36:55.930363438 +0000 UTC m=+0.395261477,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}"
Jul 22 12:36:55.993952 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:55.993928    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kcgkc"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:55.998837    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-197.ec2.internal.18c49cdace9eca63\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18c49cdace9eca63  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-07-22 12:36:55.928875619 +0000 UTC m=+0.393773665,LastTimestamp:2026-07-22 12:36:55.930391716 +0000 UTC m=+0.395289756,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.010096    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.011354    2563 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-128-197.ec2.internal.18c49cdace9efdc2\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-197.ec2.internal.18c49cdace9efdc2  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-197.ec2.internal,UID:ip-10-0-128-197.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-197.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-197.ec2.internal,},FirstTimestamp:2026-07-22 12:36:55.92888877 +0000 UTC m=+0.393786809,LastTimestamp:2026-07-22 12:36:55.930401254 +0000 UTC m=+0.395299294,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-197.ec2.internal,}"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.011778    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.011812    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.011832    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.011876    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 22 12:36:56.017575 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.014177    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-kcgkc"
Jul 22 12:36:56.024710 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.024691    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:36:56.065282 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.065213    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:56.066791 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.066773    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:56.066899 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.066809    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:56.066899 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.066823    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:56.066899 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.066853    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.080365 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.080345    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.080433 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.080368    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-197.ec2.internal\": node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.101362 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.101336    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.112870 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.112847    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"]
Jul 22 12:36:56.112928 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.112908    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:56.113867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.113849    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:56.113956 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.113886    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:56.113956 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.113900    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:56.115931 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.115914    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:56.116082 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.116129 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116097    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:56.116711 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116695    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:56.116800 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116717    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:56.116800 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116728    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:56.116800 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116734    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:56.116800 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116739    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:56.116800 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.116762    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:56.118114 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.118096    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.118200 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.118121    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:56.119410 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.119395    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:56.119515 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.119419    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:56.119515 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.119428    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:56.143505 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.143484    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-197.ec2.internal\" not found" node="ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.148057 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.148042    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-197.ec2.internal\" not found" node="ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.202222 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.202192    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.209663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.209644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b588d3c42655c06965a854981c8cfc4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"0b588d3c42655c06965a854981c8cfc4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.209815 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.209671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b588d3c42655c06965a854981c8cfc4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"0b588d3c42655c06965a854981c8cfc4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.209815 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.209689    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f81c43c62f71f9af9b24ee2595804b9b-config\") pod \"kube-apiserver-proxy-ip-10-0-128-197.ec2.internal\" (UID: \"f81c43c62f71f9af9b24ee2595804b9b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.303262 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.303223    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.310693 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.310661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f81c43c62f71f9af9b24ee2595804b9b-config\") pod \"kube-apiserver-proxy-ip-10-0-128-197.ec2.internal\" (UID: \"f81c43c62f71f9af9b24ee2595804b9b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.310814 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.310689    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f81c43c62f71f9af9b24ee2595804b9b-config\") pod \"kube-apiserver-proxy-ip-10-0-128-197.ec2.internal\" (UID: \"f81c43c62f71f9af9b24ee2595804b9b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.310814 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.310762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b588d3c42655c06965a854981c8cfc4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"0b588d3c42655c06965a854981c8cfc4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.310814 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.310793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b588d3c42655c06965a854981c8cfc4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"0b588d3c42655c06965a854981c8cfc4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.310942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.310840    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b588d3c42655c06965a854981c8cfc4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"0b588d3c42655c06965a854981c8cfc4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.310942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.310845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b588d3c42655c06965a854981c8cfc4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal\" (UID: \"0b588d3c42655c06965a854981c8cfc4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.403796 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.403688    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.447190 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.447162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.450872 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.450731    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:56.503828 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.503793    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.604278 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.604243    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.704903 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.704819    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.805320 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.805290    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.837919 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.837886    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 22 12:36:56.838502 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.838093    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 22 12:36:56.905392 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:56.905355    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:56.920607 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:56.920578    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf81c43c62f71f9af9b24ee2595804b9b.slice/crio-410e6c77e1c549bbdf83ac9f955f6b09baf1792b51cc1147ed5872ce61e82239 WatchSource:0}: Error finding container 410e6c77e1c549bbdf83ac9f955f6b09baf1792b51cc1147ed5872ce61e82239: Status 404 returned error can't find the container with id 410e6c77e1c549bbdf83ac9f955f6b09baf1792b51cc1147ed5872ce61e82239
Jul 22 12:36:56.925134 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.925115    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:36:56.989056 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.988957    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 22 12:36:56.990888 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.990857    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientMemory"
Jul 22 12:36:56.991006 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.990901    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasNoDiskPressure"
Jul 22 12:36:56.991006 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:56.990916    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeHasSufficientPID"
Jul 22 12:36:57.005730 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:57.005703    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:57.016114 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.016069    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" event={"ID":"f81c43c62f71f9af9b24ee2595804b9b","Type":"ContainerStarted","Data":"410e6c77e1c549bbdf83ac9f955f6b09baf1792b51cc1147ed5872ce61e82239"}
Jul 22 12:36:57.020502 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.020466    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 12:31:56 +0000 UTC" deadline="2028-03-31 01:45:53.189113932 +0000 UTC"
Jul 22 12:36:57.020557 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.020502    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14821h8m56.168614755s"
Jul 22 12:36:57.112121 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:57.112077    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-197.ec2.internal\" not found"
Jul 22 12:36:57.148552 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.148523    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:36:57.179412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.179360    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:36:57.206818 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.206780    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:57.222959 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.222931    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:36:57.224150 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.224135    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal"
Jul 22 12:36:57.250809 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.250734    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 22 12:36:57.515552 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.515475    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:36:57.533533 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:57.533506    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b588d3c42655c06965a854981c8cfc4.slice/crio-848a745439c2a68f828852d9f8249c8ddf9fe80decb06db1e1147ee68c3cde82 WatchSource:0}: Error finding container 848a745439c2a68f828852d9f8249c8ddf9fe80decb06db1e1147ee68c3cde82: Status 404 returned error can't find the container with id 848a745439c2a68f828852d9f8249c8ddf9fe80decb06db1e1147ee68c3cde82
Jul 22 12:36:57.878876 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.878796    2563 apiserver.go:52] "Watching apiserver"
Jul 22 12:36:57.889247 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.889209    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 22 12:36:57.890444 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.890419    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zfw59","kube-system/konnectivity-agent-qxt4l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd","openshift-dns/node-resolver-7cb9k","openshift-image-registry/node-ca-xvbww","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal","openshift-multus/multus-additional-cni-plugins-4rhfq","openshift-multus/multus-dbkm5","kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal","openshift-cluster-node-tuning-operator/tuned-j2pjz","openshift-multus/network-metrics-daemon-546pc","openshift-network-diagnostics/network-check-target-7sb8v","openshift-network-operator/iptables-alerter-4gtqw","openshift-ovn-kubernetes/ovnkube-node-6m94m"]
Jul 22 12:36:57.892099 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.892077    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:57.894089 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.894060    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:57.895594 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.895297    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:36:57.895594 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:57.895374    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:36:57.896684 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.896663    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.897849 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.897814    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:57.899029 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.899009    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:57.899175 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:57.899075    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:36:57.899663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.899640    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-knszc\""
Jul 22 12:36:57.899872 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.899851    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 22 12:36:57.900022 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.900002    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 22 12:36:57.900687 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.900667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.904274 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.904247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.905363 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.905342    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:57.905471 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:57.905418    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:36:57.905640 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.905606    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 22 12:36:57.906375 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.906346    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:36:57.906585 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.906561    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 22 12:36:57.906984 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.906967    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 22 12:36:57.907442 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.907422    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.909208 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.909193    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-cmt27\""
Jul 22 12:36:57.909390 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.909380    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 22 12:36:57.909684 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.909572    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:57.910308 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.910288    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:57.911911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.911734    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-n4fck\""
Jul 22 12:36:57.912011 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.911959    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 22 12:36:57.918469 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.918410    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 22 12:36:57.918469 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.918448    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8d9dr\""
Jul 22 12:36:57.918741 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.918720    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 22 12:36:57.919011 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.918991    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 22 12:36:57.919093 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919024    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 22 12:36:57.919093 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.918726    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 22 12:36:57.919198 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.918819    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 22 12:36:57.919198 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919030    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 22 12:36:57.919297 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919201    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:36:57.919297 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919257    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 22 12:36:57.919401 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg9v4\" (UniqueName: \"kubernetes.io/projected/c57ae273-6130-41e0-9e27-04b2cb7334c0-kube-api-access-rg9v4\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.919454 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919412    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-system-cni-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.919454 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919439    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55bbc72c-dd19-499a-b235-495b327a01f6-tmp\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.919563 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919466    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-modprobe-d\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.919563 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.919672 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-daemon-config\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.919672 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-kubelet-config\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:57.919672 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:57.919672 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919643    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-device-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.919672 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7fc6af2f-0e72-47a8-aff8-507efe413ecc-cni-binary-copy\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.919672 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-hostroot\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919695    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-conf-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919733    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5pjm\" (UniqueName: \"kubernetes.io/projected/7fc6af2f-0e72-47a8-aff8-507efe413ecc-kube-api-access-k5pjm\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc2nz\" (UniqueName: \"kubernetes.io/projected/847d1d73-3c3c-4cce-a5ce-2f05752884f5-kube-api-access-jc2nz\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-kubelet\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919842    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-os-release\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919859    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-k8s-cni-cncf-io\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-netns\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysctl-conf\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919925    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-systemd\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-run\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-cnibin\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.919997    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwdr9\" (UniqueName: \"kubernetes.io/projected/55bbc72c-dd19-499a-b235-495b327a01f6-kube-api-access-mwdr9\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-os-release\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.920127 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920059    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-socket-dir-parent\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920146    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vs2qz\" (UniqueName: \"kubernetes.io/projected/31e867fc-89b8-4fc8-a221-7ce020a315f2-kube-api-access-vs2qz\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920173    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-host\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-cni-multus\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920221    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55bbc72c-dd19-499a-b235-495b327a01f6-etc-tuned\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-dbus\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-host\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920295    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-socket-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920346    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920370    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysctl-d\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94dh7\" (UniqueName: \"kubernetes.io/projected/7172bfdb-6e7f-4586-8319-c3e9990fa020-kube-api-access-94dh7\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920421    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920445    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-etc-kubernetes\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-registration-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:36:57.920860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920529    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-system-cni-dir\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920553    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysconfig\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920575    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-var-lib-kubelet\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920595    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnbqc\" (UniqueName: \"kubernetes.io/projected/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-kube-api-access-fnbqc\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920647    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-multus-certs\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920664    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-kubernetes\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-sys\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-cnibin\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/847d1d73-3c3c-4cce-a5ce-2f05752884f5-hosts-file\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/847d1d73-3c3c-4cce-a5ce-2f05752884f5-tmp-dir\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920885    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31e867fc-89b8-4fc8-a221-7ce020a315f2-host-slash\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-lib-modules\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920954    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnl2c\" (UniqueName: \"kubernetes.io/projected/35146e94-fff6-4cb4-84e8-c53843541594-kube-api-access-fnl2c\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.921867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.920977    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-serviceca\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:57.922742 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.921003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31e867fc-89b8-4fc8-a221-7ce020a315f2-iptables-alerter-script\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:57.922742 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.921027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-sys-fs\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:57.922742 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.921058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-cni-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.922742 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.921082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-cni-bin\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923146    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923150    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923204    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923294    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923332    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4w98b\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923413    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923444    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923481    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-rffs6\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923508    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-7zg26\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923584    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923146    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923640    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923651    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xz9d6\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923640    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 22 12:36:57.923750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.923727    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-w895b\""
Jul 22 12:36:57.932603 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.932576    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 22 12:36:57.983335 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:57.983305    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-47pl7"
Jul 22 12:36:58.000061 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.000010    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-47pl7"
Jul 22 12:36:58.008528 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.008495    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 22 12:36:58.018464 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.018433    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" event={"ID":"0b588d3c42655c06965a854981c8cfc4","Type":"ContainerStarted","Data":"848a745439c2a68f828852d9f8249c8ddf9fe80decb06db1e1147ee68c3cde82"}
Jul 22 12:36:58.021893 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.021850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg9v4\" (UniqueName: \"kubernetes.io/projected/c57ae273-6130-41e0-9e27-04b2cb7334c0-kube-api-access-rg9v4\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.021893 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.021891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-system-cni-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.021918    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55bbc72c-dd19-499a-b235-495b327a01f6-tmp\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.021945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-modprobe-d\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.021972    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022000    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-daemon-config\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-kubelet-config\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022052    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022077    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-device-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7fc6af2f-0e72-47a8-aff8-507efe413ecc-cni-binary-copy\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022136 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-hostroot\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022149    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-conf-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5pjm\" (UniqueName: \"kubernetes.io/projected/7fc6af2f-0e72-47a8-aff8-507efe413ecc-kube-api-access-k5pjm\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022212    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc2nz\" (UniqueName: \"kubernetes.io/projected/847d1d73-3c3c-4cce-a5ce-2f05752884f5-kube-api-access-jc2nz\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-kubelet\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-kubelet-config\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022272    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022303    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-system-cni-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.022645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022413    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-device-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.023088 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022669    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 22 12:36:58.023088 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022868    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-hostroot\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023088 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022918    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-conf-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023088 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022954    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7fc6af2f-0e72-47a8-aff8-507efe413ecc-cni-binary-copy\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023088 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-kubelet\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.022330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-ovnkube-script-lib\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-modprobe-d\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-os-release\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-k8s-cni-cncf-io\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023239    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-netns\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023263    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysctl-conf\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-systemd\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023290    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-k8s-cni-cncf-io\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023308    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-run\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-cnibin\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-os-release\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-systemd-units\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023406    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwdr9\" (UniqueName: \"kubernetes.io/projected/55bbc72c-dd19-499a-b235-495b327a01f6-kube-api-access-mwdr9\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023431    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-os-release\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023454    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-socket-dir-parent\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-cnibin\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-netns\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023407    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-systemd\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-slash\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-daemon-config\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023526    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-run-netns\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023554    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysctl-conf\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-socket-dir-parent\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023530    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-os-release\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-systemd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.023784 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.023782    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-cni-bin\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023829    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/940957ef-c584-4bab-a90d-f7391583c9e3-ovn-node-metrics-cert\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vs2qz\" (UniqueName: \"kubernetes.io/projected/31e867fc-89b8-4fc8-a221-7ce020a315f2-kube-api-access-vs2qz\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-host\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023903    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-cni-multus\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023949    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-cni-multus\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.023957    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:36:58.523901472 +0000 UTC m=+2.988799521 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023974    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-host\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.023500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-run\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-env-overrides\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/657fc080-2e63-4659-8391-a624921d6a7b-agent-certs\") pod \"konnectivity-agent-qxt4l\" (UID: \"657fc080-2e63-4659-8391-a624921d6a7b\") " pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55bbc72c-dd19-499a-b235-495b327a01f6-etc-tuned\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024190    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-dbus\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-host\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024274    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-socket-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024333    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-host\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024482    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-etc-selinux\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-log-socket\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysctl-d\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-socket-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024585    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94dh7\" (UniqueName: \"kubernetes.io/projected/7172bfdb-6e7f-4586-8319-c3e9990fa020-kube-api-access-94dh7\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-dbus\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024612    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024654    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-etc-kubernetes\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysctl-d\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024709    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpszd\" (UniqueName: \"kubernetes.io/projected/940957ef-c584-4bab-a90d-f7391583c9e3-kube-api-access-gpszd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024729    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-etc-kubernetes\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/657fc080-2e63-4659-8391-a624921d6a7b-konnectivity-ca\") pod \"konnectivity-agent-qxt4l\" (UID: \"657fc080-2e63-4659-8391-a624921d6a7b\") " pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-registration-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.025511 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-var-lib-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-ovn\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-system-cni-dir\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysconfig\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-var-lib-kubelet\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.024977    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025006    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnbqc\" (UniqueName: \"kubernetes.io/projected/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-kube-api-access-fnbqc\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025036    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-multus-certs\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-ovnkube-config\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025091    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-registration-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025099    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-kubernetes\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-sys\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025161    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-kubernetes\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025174    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.026412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-cnibin\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.027288 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025238    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-kubelet\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.027288 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-kubelet-dir\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.027288 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.025247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-cnibin\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.027288 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-etc-sysconfig\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.027288 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027256    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-etc-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.027534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027321    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-node-log\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.027534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027380    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-cni-netd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.027534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:58.027534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027458    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/847d1d73-3c3c-4cce-a5ce-2f05752884f5-hosts-file\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:58.027534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/847d1d73-3c3c-4cce-a5ce-2f05752884f5-tmp-dir\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:58.027534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c57ae273-6130-41e0-9e27-04b2cb7334c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.027853 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/55bbc72c-dd19-499a-b235-495b327a01f6-tmp\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.027853 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-run-multus-certs\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.027964 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/847d1d73-3c3c-4cce-a5ce-2f05752884f5-tmp-dir\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:58.027964 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027924    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31e867fc-89b8-4fc8-a221-7ce020a315f2-host-slash\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:58.027964 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027957    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-lib-modules\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.028122 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.028006    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:36:58.028122 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-var-lib-kubelet\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.028122 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.027807    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c57ae273-6130-41e0-9e27-04b2cb7334c0-system-cni-dir\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.028122 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnl2c\" (UniqueName: \"kubernetes.io/projected/35146e94-fff6-4cb4-84e8-c53843541594-kube-api-access-fnl2c\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.028122 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-serviceca\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:58.028372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-lib-modules\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.028372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31e867fc-89b8-4fc8-a221-7ce020a315f2-iptables-alerter-script\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:58.028372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/31e867fc-89b8-4fc8-a221-7ce020a315f2-host-slash\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:58.028372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028228    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-sys-fs\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.028372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-cni-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.028372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028295    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-cni-bin\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.028675 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-host-var-lib-cni-bin\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.029749 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.028987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/31e867fc-89b8-4fc8-a221-7ce020a315f2-iptables-alerter-script\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:58.029749 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.029217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/35146e94-fff6-4cb4-84e8-c53843541594-sys-fs\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.029749 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.029242    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7fc6af2f-0e72-47a8-aff8-507efe413ecc-multus-cni-dir\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.029749 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.029284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/847d1d73-3c3c-4cce-a5ce-2f05752884f5-hosts-file\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:58.029749 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.029418    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret podName:9185d527-afa2-47e7-ac1b-73aae2dcbc9c nodeName:}" failed. No retries permitted until 2026-07-22 12:36:58.529398448 +0000 UTC m=+2.994296550 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret") pod "global-pull-secret-syncer-zfw59" (UID: "9185d527-afa2-47e7-ac1b-73aae2dcbc9c") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:36:58.029749 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.029633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-serviceca\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:58.029749 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.029719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/55bbc72c-dd19-499a-b235-495b327a01f6-sys\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.032962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.032926    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/55bbc72c-dd19-499a-b235-495b327a01f6-etc-tuned\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.037863 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.037757    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94dh7\" (UniqueName: \"kubernetes.io/projected/7172bfdb-6e7f-4586-8319-c3e9990fa020-kube-api-access-94dh7\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:58.037863 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.037758    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5pjm\" (UniqueName: \"kubernetes.io/projected/7fc6af2f-0e72-47a8-aff8-507efe413ecc-kube-api-access-k5pjm\") pod \"multus-dbkm5\" (UID: \"7fc6af2f-0e72-47a8-aff8-507efe413ecc\") " pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.038757 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.038696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg9v4\" (UniqueName: \"kubernetes.io/projected/c57ae273-6130-41e0-9e27-04b2cb7334c0-kube-api-access-rg9v4\") pod \"multus-additional-cni-plugins-4rhfq\" (UID: \"c57ae273-6130-41e0-9e27-04b2cb7334c0\") " pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.040464 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.040444    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwdr9\" (UniqueName: \"kubernetes.io/projected/55bbc72c-dd19-499a-b235-495b327a01f6-kube-api-access-mwdr9\") pod \"tuned-j2pjz\" (UID: \"55bbc72c-dd19-499a-b235-495b327a01f6\") " pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.040699 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.040676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vs2qz\" (UniqueName: \"kubernetes.io/projected/31e867fc-89b8-4fc8-a221-7ce020a315f2-kube-api-access-vs2qz\") pod \"iptables-alerter-4gtqw\" (UID: \"31e867fc-89b8-4fc8-a221-7ce020a315f2\") " pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:58.040784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.040676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnbqc\" (UniqueName: \"kubernetes.io/projected/dba8b7c7-8dd2-4913-ab86-8e65582cb93f-kube-api-access-fnbqc\") pod \"node-ca-xvbww\" (UID: \"dba8b7c7-8dd2-4913-ab86-8e65582cb93f\") " pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:58.042914 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.042891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc2nz\" (UniqueName: \"kubernetes.io/projected/847d1d73-3c3c-4cce-a5ce-2f05752884f5-kube-api-access-jc2nz\") pod \"node-resolver-7cb9k\" (UID: \"847d1d73-3c3c-4cce-a5ce-2f05752884f5\") " pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:58.043116 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.043096    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 22 12:36:58.046366 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.046343    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:36:58.046366 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.046367    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:36:58.046523 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.046377    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cfvcq for pod openshift-network-diagnostics/network-check-target-7sb8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:36:58.046523 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.046439    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq podName:1ecb27d5-2560-44dc-8ce0-992c6aa48e51 nodeName:}" failed. No retries permitted until 2026-07-22 12:36:58.546424739 +0000 UTC m=+3.011322767 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-cfvcq" (UniqueName: "kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq") pod "network-check-target-7sb8v" (UID: "1ecb27d5-2560-44dc-8ce0-992c6aa48e51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:36:58.053132 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.053103    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnl2c\" (UniqueName: \"kubernetes.io/projected/35146e94-fff6-4cb4-84e8-c53843541594-kube-api-access-fnl2c\") pod \"aws-ebs-csi-driver-node-zjpfd\" (UID: \"35146e94-fff6-4cb4-84e8-c53843541594\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.129430 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129333    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129430 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129430 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129396    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-ovnkube-script-lib\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-run-ovn-kubernetes\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-systemd-units\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129501    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-slash\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-slash\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-run-netns\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-systemd-units\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-systemd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-run-netns\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129614    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-cni-bin\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129640    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/940957ef-c584-4bab-a90d-f7391583c9e3-ovn-node-metrics-cert\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129668    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-env-overrides\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129675    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-cni-bin\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-systemd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.129723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/657fc080-2e63-4659-8391-a624921d6a7b-agent-certs\") pod \"konnectivity-agent-qxt4l\" (UID: \"657fc080-2e63-4659-8391-a624921d6a7b\") " pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129735    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-log-socket\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129764    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129813    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpszd\" (UniqueName: \"kubernetes.io/projected/940957ef-c584-4bab-a90d-f7391583c9e3-kube-api-access-gpszd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/657fc080-2e63-4659-8391-a624921d6a7b-konnectivity-ca\") pod \"konnectivity-agent-qxt4l\" (UID: \"657fc080-2e63-4659-8391-a624921d6a7b\") " pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-var-lib-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-ovn\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129937    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-ovnkube-config\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-kubelet\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.129988    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-etc-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-node-log\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-cni-netd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-cni-netd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130212    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-ovnkube-script-lib\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.130413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130267    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-ovn\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-host-kubelet\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-var-lib-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/657fc080-2e63-4659-8391-a624921d6a7b-konnectivity-ca\") pod \"konnectivity-agent-qxt4l\" (UID: \"657fc080-2e63-4659-8391-a624921d6a7b\") " pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130632    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-log-socket\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130638    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-node-log\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130694    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-run-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.130971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/940957ef-c584-4bab-a90d-f7391583c9e3-etc-openvswitch\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.131010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-env-overrides\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.131434 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.131403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/940957ef-c584-4bab-a90d-f7391583c9e3-ovnkube-config\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.132749 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.132721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/940957ef-c584-4bab-a90d-f7391583c9e3-ovn-node-metrics-cert\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.134069 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.134050    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/657fc080-2e63-4659-8391-a624921d6a7b-agent-certs\") pod \"konnectivity-agent-qxt4l\" (UID: \"657fc080-2e63-4659-8391-a624921d6a7b\") " pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:58.145179 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.145150    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpszd\" (UniqueName: \"kubernetes.io/projected/940957ef-c584-4bab-a90d-f7391583c9e3-kube-api-access-gpszd\") pod \"ovnkube-node-6m94m\" (UID: \"940957ef-c584-4bab-a90d-f7391583c9e3\") " pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.206369 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.206328    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-7cb9k"
Jul 22 12:36:58.215851 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.215823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4gtqw"
Jul 22 12:36:58.217102 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.217076    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod847d1d73_3c3c_4cce_a5ce_2f05752884f5.slice/crio-5d9307e204f939e1d6dc2338781103a302429ac91feab921e61a0d8e6aab80a1 WatchSource:0}: Error finding container 5d9307e204f939e1d6dc2338781103a302429ac91feab921e61a0d8e6aab80a1: Status 404 returned error can't find the container with id 5d9307e204f939e1d6dc2338781103a302429ac91feab921e61a0d8e6aab80a1
Jul 22 12:36:58.224657 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.224629    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e867fc_89b8_4fc8_a221_7ce020a315f2.slice/crio-2f5620e4cca513adcf37429614c70e31b70c707bf429fcf21fd52af1c03d1e4c WatchSource:0}: Error finding container 2f5620e4cca513adcf37429614c70e31b70c707bf429fcf21fd52af1c03d1e4c: Status 404 returned error can't find the container with id 2f5620e4cca513adcf37429614c70e31b70c707bf429fcf21fd52af1c03d1e4c
Jul 22 12:36:58.225849 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.225767    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4rhfq"
Jul 22 12:36:58.233162 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.233137    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-xvbww"
Jul 22 12:36:58.234859 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.234836    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc57ae273_6130_41e0_9e27_04b2cb7334c0.slice/crio-d83b6953624888443e86f39af15470e2db2ad313c3aadc53ec859d93feead190 WatchSource:0}: Error finding container d83b6953624888443e86f39af15470e2db2ad313c3aadc53ec859d93feead190: Status 404 returned error can't find the container with id d83b6953624888443e86f39af15470e2db2ad313c3aadc53ec859d93feead190
Jul 22 12:36:58.241315 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.241276    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dbkm5"
Jul 22 12:36:58.241615 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.241589    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddba8b7c7_8dd2_4913_ab86_8e65582cb93f.slice/crio-95b14a77cde32b858128e5f97022aaa76f7f0cf6bec2761eb36a162271cec921 WatchSource:0}: Error finding container 95b14a77cde32b858128e5f97022aaa76f7f0cf6bec2761eb36a162271cec921: Status 404 returned error can't find the container with id 95b14a77cde32b858128e5f97022aaa76f7f0cf6bec2761eb36a162271cec921
Jul 22 12:36:58.249750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.249729    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-j2pjz"
Jul 22 12:36:58.250743 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.250722    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fc6af2f_0e72_47a8_aff8_507efe413ecc.slice/crio-0dd0ea49eb2e6ae77ebc81ba70cb35928350ea97109eea6a5d3560933db2635f WatchSource:0}: Error finding container 0dd0ea49eb2e6ae77ebc81ba70cb35928350ea97109eea6a5d3560933db2635f: Status 404 returned error can't find the container with id 0dd0ea49eb2e6ae77ebc81ba70cb35928350ea97109eea6a5d3560933db2635f
Jul 22 12:36:58.257064 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.257042    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd"
Jul 22 12:36:58.257788 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.257757    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55bbc72c_dd19_499a_b235_495b327a01f6.slice/crio-c5c1bbf8bf3615656cffd5707a1e21e64ee20d8535c180e6a87b0ca1c6fc9f28 WatchSource:0}: Error finding container c5c1bbf8bf3615656cffd5707a1e21e64ee20d8535c180e6a87b0ca1c6fc9f28: Status 404 returned error can't find the container with id c5c1bbf8bf3615656cffd5707a1e21e64ee20d8535c180e6a87b0ca1c6fc9f28
Jul 22 12:36:58.265963 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.265941    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:36:58.266664 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.266640    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35146e94_fff6_4cb4_84e8_c53843541594.slice/crio-c157589aece7c0703373bbbe357526455cfd7444f6eb0201a805852bdf09bd6d WatchSource:0}: Error finding container c157589aece7c0703373bbbe357526455cfd7444f6eb0201a805852bdf09bd6d: Status 404 returned error can't find the container with id c157589aece7c0703373bbbe357526455cfd7444f6eb0201a805852bdf09bd6d
Jul 22 12:36:58.273957 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.273939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:36:58.274471 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.274445    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod940957ef_c584_4bab_a90d_f7391583c9e3.slice/crio-c77723dc65080d5b38a703354b6f94129ebd570dc6d7b869207f4e69d7398a3e WatchSource:0}: Error finding container c77723dc65080d5b38a703354b6f94129ebd570dc6d7b869207f4e69d7398a3e: Status 404 returned error can't find the container with id c77723dc65080d5b38a703354b6f94129ebd570dc6d7b869207f4e69d7398a3e
Jul 22 12:36:58.282707 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:36:58.282665    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657fc080_2e63_4659_8391_a624921d6a7b.slice/crio-1023a5977621203c0b2d9078c1186f6d3ced90f8852d1a79cac7047538c7e9de WatchSource:0}: Error finding container 1023a5977621203c0b2d9078c1186f6d3ced90f8852d1a79cac7047538c7e9de: Status 404 returned error can't find the container with id 1023a5977621203c0b2d9078c1186f6d3ced90f8852d1a79cac7047538c7e9de
Jul 22 12:36:58.533962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.533862    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:58.533962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.533928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:58.534179 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.534059    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:36:58.534179 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.534065    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:36:58.534179 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.534138    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:36:59.534115789 +0000 UTC m=+3.999013861 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:36:58.534179 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.534161    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret podName:9185d527-afa2-47e7-ac1b-73aae2dcbc9c nodeName:}" failed. No retries permitted until 2026-07-22 12:36:59.534150465 +0000 UTC m=+3.999048495 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret") pod "global-pull-secret-syncer-zfw59" (UID: "9185d527-afa2-47e7-ac1b-73aae2dcbc9c") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:36:58.634942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:58.634902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:36:58.635147 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.635124    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:36:58.635200 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.635153    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:36:58.635200 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.635165    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cfvcq for pod openshift-network-diagnostics/network-check-target-7sb8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:36:58.635283 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:58.635218    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq podName:1ecb27d5-2560-44dc-8ce0-992c6aa48e51 nodeName:}" failed. No retries permitted until 2026-07-22 12:36:59.635203 +0000 UTC m=+4.100101027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-cfvcq" (UniqueName: "kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq") pod "network-check-target-7sb8v" (UID: "1ecb27d5-2560-44dc-8ce0-992c6aa48e51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:36:59.002493 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.002425    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 12:31:58 +0000 UTC" deadline="2028-05-02 05:56:44.374189928 +0000 UTC"
Jul 22 12:36:59.002493 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.002490    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15593h19m45.371704842s"
Jul 22 12:36:59.028469 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.028355    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7cb9k" event={"ID":"847d1d73-3c3c-4cce-a5ce-2f05752884f5","Type":"ContainerStarted","Data":"5d9307e204f939e1d6dc2338781103a302429ac91feab921e61a0d8e6aab80a1"}
Jul 22 12:36:59.030264 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.030186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qxt4l" event={"ID":"657fc080-2e63-4659-8391-a624921d6a7b","Type":"ContainerStarted","Data":"1023a5977621203c0b2d9078c1186f6d3ced90f8852d1a79cac7047538c7e9de"}
Jul 22 12:36:59.032265 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.032152    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"c77723dc65080d5b38a703354b6f94129ebd570dc6d7b869207f4e69d7398a3e"}
Jul 22 12:36:59.034717 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.034684    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd" event={"ID":"35146e94-fff6-4cb4-84e8-c53843541594","Type":"ContainerStarted","Data":"c157589aece7c0703373bbbe357526455cfd7444f6eb0201a805852bdf09bd6d"}
Jul 22 12:36:59.036307 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.036284    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j2pjz" event={"ID":"55bbc72c-dd19-499a-b235-495b327a01f6","Type":"ContainerStarted","Data":"c5c1bbf8bf3615656cffd5707a1e21e64ee20d8535c180e6a87b0ca1c6fc9f28"}
Jul 22 12:36:59.037980 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.037942    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dbkm5" event={"ID":"7fc6af2f-0e72-47a8-aff8-507efe413ecc","Type":"ContainerStarted","Data":"0dd0ea49eb2e6ae77ebc81ba70cb35928350ea97109eea6a5d3560933db2635f"}
Jul 22 12:36:59.039960 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.039836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xvbww" event={"ID":"dba8b7c7-8dd2-4913-ab86-8e65582cb93f","Type":"ContainerStarted","Data":"95b14a77cde32b858128e5f97022aaa76f7f0cf6bec2761eb36a162271cec921"}
Jul 22 12:36:59.041293 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.041269    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerStarted","Data":"d83b6953624888443e86f39af15470e2db2ad313c3aadc53ec859d93feead190"}
Jul 22 12:36:59.042726 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.042683    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4gtqw" event={"ID":"31e867fc-89b8-4fc8-a221-7ce020a315f2","Type":"ContainerStarted","Data":"2f5620e4cca513adcf37429614c70e31b70c707bf429fcf21fd52af1c03d1e4c"}
Jul 22 12:36:59.546023 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.544783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:36:59.546023 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.545098    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:36:59.546023 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.545299    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:36:59.546023 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.545370    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:01.545349905 +0000 UTC m=+6.010247949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:36:59.546023 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.545842    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:36:59.546023 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.545900    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret podName:9185d527-afa2-47e7-ac1b-73aae2dcbc9c nodeName:}" failed. No retries permitted until 2026-07-22 12:37:01.545883447 +0000 UTC m=+6.010781491 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret") pod "global-pull-secret-syncer-zfw59" (UID: "9185d527-afa2-47e7-ac1b-73aae2dcbc9c") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:36:59.646777 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:36:59.646150    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:36:59.646777 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.646331    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:36:59.646777 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.646351    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:36:59.646777 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.646365    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cfvcq for pod openshift-network-diagnostics/network-check-target-7sb8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:36:59.646777 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:36:59.646425    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq podName:1ecb27d5-2560-44dc-8ce0-992c6aa48e51 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:01.646406315 +0000 UTC m=+6.111304368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-cfvcq" (UniqueName: "kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq") pod "network-check-target-7sb8v" (UID: "1ecb27d5-2560-44dc-8ce0-992c6aa48e51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:00.003081 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:00.002991    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 12:31:58 +0000 UTC" deadline="2028-01-12 21:03:04.283652052 +0000 UTC"
Jul 22 12:37:00.003081 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:00.003033    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12944h26m4.280625072s"
Jul 22 12:37:00.015056 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:00.015026    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:00.015206 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:00.015178    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:00.015657 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:00.015636    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:00.015772 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:00.015747    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:00.015997 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:00.015979    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:00.016117 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:00.016093    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:00.052527 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:00.052496    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" event={"ID":"f81c43c62f71f9af9b24ee2595804b9b","Type":"ContainerStarted","Data":"8be0326a1c4d4c5d773f57f337e38826e4e3f323763b2afadc5c4bbf40d10acc"}
Jul 22 12:37:00.071172 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:00.071098    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-197.ec2.internal" podStartSLOduration=3.07107996 podStartE2EDuration="3.07107996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:00.069967469 +0000 UTC m=+4.534865525" watchObservedRunningTime="2026-07-22 12:37:00.07107996 +0000 UTC m=+4.535978019"
Jul 22 12:37:01.561750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:01.560949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:01.561750 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:01.561030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:01.561750 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.561183    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:01.561750 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.561253    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:05.561231867 +0000 UTC m=+10.026129944 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:01.561750 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.561667    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:01.561750 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.561716    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret podName:9185d527-afa2-47e7-ac1b-73aae2dcbc9c nodeName:}" failed. No retries permitted until 2026-07-22 12:37:05.561701075 +0000 UTC m=+10.026599108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret") pod "global-pull-secret-syncer-zfw59" (UID: "9185d527-afa2-47e7-ac1b-73aae2dcbc9c") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:01.661382 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:01.661342    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:01.661588 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.661565    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:37:01.661660 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.661597    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:37:01.661660 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.661611    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cfvcq for pod openshift-network-diagnostics/network-check-target-7sb8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:01.661760 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:01.661682    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq podName:1ecb27d5-2560-44dc-8ce0-992c6aa48e51 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:05.661650451 +0000 UTC m=+10.126548498 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-cfvcq" (UniqueName: "kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq") pod "network-check-target-7sb8v" (UID: "1ecb27d5-2560-44dc-8ce0-992c6aa48e51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:02.013014 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:02.012870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:02.013146 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:02.013016    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:02.013415 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:02.013394    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:02.013535 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:02.013510    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:02.013684 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:02.013667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:02.013836 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:02.013802    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:04.013451 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:04.012880    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:04.013451 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:04.013009    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:04.013451 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:04.013049    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:04.013451 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:04.013042    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:04.013451 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:04.013155    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:04.013451 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:04.013245    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:05.069477 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:05.069425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" event={"ID":"0b588d3c42655c06965a854981c8cfc4","Type":"ContainerStarted","Data":"e4ec1cfc523702efa1ac03ca2f51886f655cc780e14770526dffd86681528989"}
Jul 22 12:37:05.607857 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:05.607187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:05.607857 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:05.607238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:05.607857 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.607371    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:05.607857 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.607451    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:13.607418377 +0000 UTC m=+18.072316430 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:05.608197 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.607906    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:05.608197 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.607965    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret podName:9185d527-afa2-47e7-ac1b-73aae2dcbc9c nodeName:}" failed. No retries permitted until 2026-07-22 12:37:13.607948296 +0000 UTC m=+18.072846339 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret") pod "global-pull-secret-syncer-zfw59" (UID: "9185d527-afa2-47e7-ac1b-73aae2dcbc9c") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:05.709061 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:05.708372    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:05.709061 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.708560    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:37:05.709061 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.708582    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:37:05.709061 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.708596    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cfvcq for pod openshift-network-diagnostics/network-check-target-7sb8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:05.709061 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:05.708655    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq podName:1ecb27d5-2560-44dc-8ce0-992c6aa48e51 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:13.708635381 +0000 UTC m=+18.173533417 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-cfvcq" (UniqueName: "kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq") pod "network-check-target-7sb8v" (UID: "1ecb27d5-2560-44dc-8ce0-992c6aa48e51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:06.013990 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:06.013900    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:06.014166 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:06.014024    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:06.014166 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:06.014045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:06.014166 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:06.014050    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:06.014314 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:06.014170    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:06.014314 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:06.014272    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:08.014507 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:08.014436    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:08.014983 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:08.014536    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:08.014983 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:08.014881    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:08.014983 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:08.014889    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:08.014983 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:08.014958    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:08.015220 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:08.015023    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:08.079558 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:08.079525    2563 generic.go:356] "Generic (PLEG): container finished" podID="0b588d3c42655c06965a854981c8cfc4" containerID="e4ec1cfc523702efa1ac03ca2f51886f655cc780e14770526dffd86681528989" exitCode=0
Jul 22 12:37:08.079772 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:08.079581    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" event={"ID":"0b588d3c42655c06965a854981c8cfc4","Type":"ContainerDied","Data":"e4ec1cfc523702efa1ac03ca2f51886f655cc780e14770526dffd86681528989"}
Jul 22 12:37:10.012872 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:10.012846    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:10.013286 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:10.012845    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:10.013286 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:10.012954    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:10.013286 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:10.012845    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:10.013286 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:10.013019    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:10.013286 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:10.013124    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:12.012632 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:12.012599    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:12.013044 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:12.012612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:12.013044 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:12.012731    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:12.013044 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:12.012727    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:12.013044 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:12.012845    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:12.013044 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:12.012914    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:13.670315 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:13.670286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:13.670786 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:13.670367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:13.670786 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.670460    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:13.670786 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.670460    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:13.670786 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.670513    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret podName:9185d527-afa2-47e7-ac1b-73aae2dcbc9c nodeName:}" failed. No retries permitted until 2026-07-22 12:37:29.67049947 +0000 UTC m=+34.135397497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret") pod "global-pull-secret-syncer-zfw59" (UID: "9185d527-afa2-47e7-ac1b-73aae2dcbc9c") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:13.670786 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.670532    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:29.67052428 +0000 UTC m=+34.135422308 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:13.771050 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:13.770993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:13.771197 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.771159    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:37:13.771197 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.771176    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:37:13.771197 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.771188    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cfvcq for pod openshift-network-diagnostics/network-check-target-7sb8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:13.771320 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:13.771248    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq podName:1ecb27d5-2560-44dc-8ce0-992c6aa48e51 nodeName:}" failed. No retries permitted until 2026-07-22 12:37:29.77123056 +0000 UTC m=+34.236128592 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-cfvcq" (UniqueName: "kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq") pod "network-check-target-7sb8v" (UID: "1ecb27d5-2560-44dc-8ce0-992c6aa48e51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:14.012502 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:14.012415    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:14.012649 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:14.012420    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:14.012649 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:14.012542    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:14.012649 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:14.012630    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:14.012829 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:14.012417    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:14.012829 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:14.012778    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:16.013320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:16.013270    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:16.013774 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:16.013361    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:16.013774 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:16.013421    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:16.013774 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:16.013488    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:16.013774 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:16.013524    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:16.014014 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:16.013970    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:18.012832 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:18.012803    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:18.012832 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:18.012802    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:18.013512 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:18.012802    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:18.013512 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:18.012931    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:18.013512 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:18.013295    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:18.013512 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:18.013359    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:19.099985 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.099334    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-xvbww" event={"ID":"dba8b7c7-8dd2-4913-ab86-8e65582cb93f","Type":"ContainerStarted","Data":"34a2873159b06880c287adbb6291fc47aef81937f5540a2d98c3e4c0a228f3ca"}
Jul 22 12:37:19.101164 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.101137    2563 generic.go:356] "Generic (PLEG): container finished" podID="c57ae273-6130-41e0-9e27-04b2cb7334c0" containerID="92fc52e1f0e1c7a03b0a4e55aaf929739a3418707d20c2bef37f60c5f3365135" exitCode=0
Jul 22 12:37:19.101280 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.101216    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerDied","Data":"92fc52e1f0e1c7a03b0a4e55aaf929739a3418707d20c2bef37f60c5f3365135"}
Jul 22 12:37:19.102829 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.102801    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-7cb9k" event={"ID":"847d1d73-3c3c-4cce-a5ce-2f05752884f5","Type":"ContainerStarted","Data":"c02a0ef74a6c43feb32f99a53dcbf65af0e95823db48ac65311dee5482e75cf2"}
Jul 22 12:37:19.104963 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.104943    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" event={"ID":"0b588d3c42655c06965a854981c8cfc4","Type":"ContainerStarted","Data":"c05a03fc2b0964f93b8833edbc82c48777946f1fc640fd0a84a451025561af54"}
Jul 22 12:37:19.106462 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.106435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qxt4l" event={"ID":"657fc080-2e63-4659-8391-a624921d6a7b","Type":"ContainerStarted","Data":"6af70552e56855b006e11fa460fc9237ca56539de1de4f8d9a11f67d5af9b4c8"}
Jul 22 12:37:19.109173 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.109151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"3e1467ddae342c8aa98db5b747439f1bf74f44439a473ef849fb78840be1a601"}
Jul 22 12:37:19.109269 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.109214    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"07de0a1fc5f2226cb5d27c8a8d732d19fd3d93ce018a56f0475d5f9b1588df71"}
Jul 22 12:37:19.109269 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.109249    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"c106fab42fb1338e184b480d4d3d8ce1fb3bf6780e2b01eba194abc7b8d1f993"}
Jul 22 12:37:19.109349 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.109276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"96ca2560f69692a7a3aec666ad62bf064c84fd25c882bffad68f71f120c4d185"}
Jul 22 12:37:19.109349 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.109315    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"d145980c437a2fe7b842198737cee819dc0728fd6db5679b03c5677b90d23a07"}
Jul 22 12:37:19.110313 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.110295    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd" event={"ID":"35146e94-fff6-4cb4-84e8-c53843541594","Type":"ContainerStarted","Data":"1d1a46a39d7853486a334e109f90e2aee82f88b7b482abd10e48b7a2675bb29d"}
Jul 22 12:37:19.111490 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.111474    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-j2pjz" event={"ID":"55bbc72c-dd19-499a-b235-495b327a01f6","Type":"ContainerStarted","Data":"25d525291efa091f349b0f6caffe097e2fe84f108ef996fb74555ca1bf3f12cf"}
Jul 22 12:37:19.112633 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.112614    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dbkm5" event={"ID":"7fc6af2f-0e72-47a8-aff8-507efe413ecc","Type":"ContainerStarted","Data":"38d2ef3e889a56a989f5c0ba599871da530a6e37f1d4238af79a674cd4124aef"}
Jul 22 12:37:19.117929 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.117895    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-xvbww" podStartSLOduration=1.9533523910000001 podStartE2EDuration="22.117883937s" totalImagesPullingTime="20.164531546s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:19.117452715 +0000 UTC m=+23.582350765" watchObservedRunningTime="2026-07-22 12:37:19.117883937 +0000 UTC m=+23.582781985"
Jul 22 12:37:19.134780 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.134712    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-197.ec2.internal" podStartSLOduration=22.134694823 podStartE2EDuration="22.134694823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:19.134054678 +0000 UTC m=+23.598952728" watchObservedRunningTime="2026-07-22 12:37:19.134694823 +0000 UTC m=+23.599592872"
Jul 22 12:37:19.156390 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.156357    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-7cb9k" podStartSLOduration=2.96348816 podStartE2EDuration="23.156345937s" totalImagesPullingTime="20.192857777s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:19.155895422 +0000 UTC m=+23.620793471" watchObservedRunningTime="2026-07-22 12:37:19.156345937 +0000 UTC m=+23.621243986"
Jul 22 12:37:19.175149 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.175084    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dbkm5" podStartSLOduration=1.955947379 podStartE2EDuration="22.175074231s" totalImagesPullingTime="20.219126852s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:19.174920116 +0000 UTC m=+23.639818165" watchObservedRunningTime="2026-07-22 12:37:19.175074231 +0000 UTC m=+23.639972279"
Jul 22 12:37:19.240966 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.240915    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-j2pjz" podStartSLOduration=2.061276165 podStartE2EDuration="22.240901677s" totalImagesPullingTime="20.179625512s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:19.220168483 +0000 UTC m=+23.685066529" watchObservedRunningTime="2026-07-22 12:37:19.240901677 +0000 UTC m=+23.705799719"
Jul 22 12:37:19.241269 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:19.241249    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qxt4l" podStartSLOduration=2.114286645 podStartE2EDuration="22.241241177s" totalImagesPullingTime="20.126954532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:19.240977336 +0000 UTC m=+23.705875387" watchObservedRunningTime="2026-07-22 12:37:19.241241177 +0000 UTC m=+23.706139226"
Jul 22 12:37:20.012378 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.012310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:20.012538 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.012317    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:20.012538 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:20.012416    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:20.012610 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:20.012539    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:20.012610 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.012576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:20.012674 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:20.012656    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:20.115504 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.115477    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4gtqw" event={"ID":"31e867fc-89b8-4fc8-a221-7ce020a315f2","Type":"ContainerStarted","Data":"57fef69003a13f17c69c50b3eb10cf880ecbc9f6a1bc6f8f817af2a204d53165"}
Jul 22 12:37:20.118264 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.118234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"e419b6d2449b1725bfd17b91f1d0f9d5ad1b19ba440f4e9f0f85cccd138cc2e3"}
Jul 22 12:37:20.131734 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.131698    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4gtqw" podStartSLOduration=3.949641993 podStartE2EDuration="24.131686728s" totalImagesPullingTime="20.182044735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:20.131369113 +0000 UTC m=+24.596267162" watchObservedRunningTime="2026-07-22 12:37:20.131686728 +0000 UTC m=+24.596584776"
Jul 22 12:37:20.359534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.359506    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 22 12:37:20.984838 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.984732    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T12:37:20.359527052Z","UUID":"4c9a0071-fbba-4191-a8ce-a028dfb17a3e","Handler":null,"Name":"","Endpoint":""}
Jul 22 12:37:20.986529 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.986507    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 22 12:37:20.986708 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:20.986545    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 22 12:37:21.122884 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:21.122851    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd" event={"ID":"35146e94-fff6-4cb4-84e8-c53843541594","Type":"ContainerStarted","Data":"a230e7d9638c0eeadeb422cb04667dc680c179776c67a7315ab2bf9268fe1925"}
Jul 22 12:37:22.014849 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:22.012845    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:22.014849 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:22.012850    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:22.014849 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:22.012967    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:22.014849 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:22.013029    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:22.014849 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:22.012854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:22.014849 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:22.013133    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:22.130913 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:22.130841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"c3fcd2fe4be47a73005a3cea08b0164f0b62a196fdb60640fe745837674716e0"}
Jul 22 12:37:22.132661 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:22.132640    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd" event={"ID":"35146e94-fff6-4cb4-84e8-c53843541594","Type":"ContainerStarted","Data":"b4e26a012cc22afbae45338ba631ffda616b0af9bc31d158549c29949ff7212a"}
Jul 22 12:37:23.161918 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:23.161652    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:37:23.162362 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:23.162239    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:37:23.177720 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:23.177682    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-zjpfd" podStartSLOduration=7.119719158 podStartE2EDuration="26.177667244s" totalImagesPullingTime="19.057948086s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:22.152564064 +0000 UTC m=+26.617462113" watchObservedRunningTime="2026-07-22 12:37:23.177667244 +0000 UTC m=+27.642565293"
Jul 22 12:37:24.012380 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:24.012344    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:24.012558 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:24.012344    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:24.012558 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:24.012480    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:24.012674 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:24.012561    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:24.012674 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:24.012349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:24.012786 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:24.012679    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:24.137775 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:24.137685    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:37:24.138192 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:24.138161    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qxt4l"
Jul 22 12:37:25.143469 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:25.143231    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" event={"ID":"940957ef-c584-4bab-a90d-f7391583c9e3","Type":"ContainerStarted","Data":"adaaaeb14594efcbc65efb40ac1c3ad0745d357344d69285e73572f6330312da"}
Jul 22 12:37:25.144099 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:25.143494    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:37:25.144099 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:25.143546    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:37:25.145080 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:25.145059    2563 generic.go:356] "Generic (PLEG): container finished" podID="c57ae273-6130-41e0-9e27-04b2cb7334c0" containerID="0f27d321151d54b0c6072e70a2c1cd80243c233d6dc5bd515904e9c431b3d61d" exitCode=0
Jul 22 12:37:25.145188 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:25.145138    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerDied","Data":"0f27d321151d54b0c6072e70a2c1cd80243c233d6dc5bd515904e9c431b3d61d"}
Jul 22 12:37:25.159266 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:25.159248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:37:25.197676 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:25.197637    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m" podStartSLOduration=7.995147635 podStartE2EDuration="28.197609209s" totalImagesPullingTime="20.202461574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:25.197312948 +0000 UTC m=+29.662211009" watchObservedRunningTime="2026-07-22 12:37:25.197609209 +0000 UTC m=+29.662507286"
Jul 22 12:37:26.013769 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.013720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:26.013769 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.013835    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:26.014505 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:26.013835    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:26.014505 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.013973    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:26.014505 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:26.014000    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:26.014505 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:26.014045    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:26.147477 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.147452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:37:26.161492 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.161468    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:37:26.709939 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.709867    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zfw59"]
Jul 22 12:37:26.710073 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.709988    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:26.710124 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:26.710075    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:26.713866 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.713842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7sb8v"]
Jul 22 12:37:26.714005 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.713939    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:26.714070 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:26.714028    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:26.714496 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.714474    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-546pc"]
Jul 22 12:37:26.714602 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:26.714566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:26.714683 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:26.714662    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:27.151671 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:27.151592    2563 generic.go:356] "Generic (PLEG): container finished" podID="c57ae273-6130-41e0-9e27-04b2cb7334c0" containerID="df642126d62e7aa60f0d04c1d79db9ed40d44bee271a936d855e02ce4aed94fc" exitCode=0
Jul 22 12:37:27.151671 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:27.151658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerDied","Data":"df642126d62e7aa60f0d04c1d79db9ed40d44bee271a936d855e02ce4aed94fc"}
Jul 22 12:37:28.012093 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:28.012068    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:28.012093 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:28.012099    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:28.012323 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:28.012201    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:28.012323 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:28.012266    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:28.012417 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:28.012323    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:28.012453 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:28.012417    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:29.157517 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:29.157348    2563 generic.go:356] "Generic (PLEG): container finished" podID="c57ae273-6130-41e0-9e27-04b2cb7334c0" containerID="2ffa2861c21a88e99c4d9dfc50615ac0fcd1f03fa80dcbfdd0fd56f590d8b1f0" exitCode=0
Jul 22 12:37:29.157517 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:29.157431    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerDied","Data":"2ffa2861c21a88e99c4d9dfc50615ac0fcd1f03fa80dcbfdd0fd56f590d8b1f0"}
Jul 22 12:37:29.690817 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:29.690779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:29.690817 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:29.690827    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:29.691012 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.690928    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:29.691012 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.690990    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret podName:9185d527-afa2-47e7-ac1b-73aae2dcbc9c nodeName:}" failed. No retries permitted until 2026-07-22 12:38:01.690976298 +0000 UTC m=+66.155874325 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret") pod "global-pull-secret-syncer-zfw59" (UID: "9185d527-afa2-47e7-ac1b-73aae2dcbc9c") : object "kube-system"/"original-pull-secret" not registered
Jul 22 12:37:29.691012 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.690928    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:29.691116 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.691044    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:38:01.691031988 +0000 UTC m=+66.155930015 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 22 12:37:29.791533 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:29.791497    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:29.791694 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.791646    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 22 12:37:29.791694 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.791666    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 22 12:37:29.791694 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.791674    2563 projected.go:196] Error preparing data for projected volume kube-api-access-cfvcq for pod openshift-network-diagnostics/network-check-target-7sb8v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:29.791825 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:29.791719    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq podName:1ecb27d5-2560-44dc-8ce0-992c6aa48e51 nodeName:}" failed. No retries permitted until 2026-07-22 12:38:01.791707118 +0000 UTC m=+66.256605144 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-cfvcq" (UniqueName: "kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq") pod "network-check-target-7sb8v" (UID: "1ecb27d5-2560-44dc-8ce0-992c6aa48e51") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 22 12:37:30.012050 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:30.012024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:30.012175 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:30.012117    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-546pc" podUID="7172bfdb-6e7f-4586-8319-c3e9990fa020"
Jul 22 12:37:30.012175 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:30.012024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:30.012297 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:30.012174    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zfw59" podUID="9185d527-afa2-47e7-ac1b-73aae2dcbc9c"
Jul 22 12:37:30.012297 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:30.012192    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:30.012392 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:30.012338    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-7sb8v" podUID="1ecb27d5-2560-44dc-8ce0-992c6aa48e51"
Jul 22 12:37:31.361206 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.361131    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-197.ec2.internal" event="NodeReady"
Jul 22 12:37:31.361690 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.361276    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 22 12:37:31.404124 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.404095    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-d7cdd955f-l8kmv"]
Jul 22 12:37:31.405970 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.405951    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.410280 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.410259    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 22 12:37:31.410376 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.410262    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-69gxc\""
Jul 22 12:37:31.410376 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.410299    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 22 12:37:31.410544 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.410527    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 22 12:37:31.420045 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.420021    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 22 12:37:31.424174 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.424154    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-d7cdd955f-l8kmv"]
Jul 22 12:37:31.427590 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.427571    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-spfbs"]
Jul 22 12:37:31.429278 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.429243    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.432182 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.432154    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-twkmz"]
Jul 22 12:37:31.432295 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.432276    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ntb22\""
Jul 22 12:37:31.432515 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.432498    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 22 12:37:31.432761 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.432732    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 22 12:37:31.434091 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.434073    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.440809 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.440793    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 22 12:37:31.441270 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.441254    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 22 12:37:31.441502 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.441483    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 22 12:37:31.441845 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.441830    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-z644t\""
Jul 22 12:37:31.451894 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.451842    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-spfbs"]
Jul 22 12:37:31.454651 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.454548    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-twkmz"]
Jul 22 12:37:31.505341 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505320    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-installation-pull-secrets\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.505479 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-certificates\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.505479 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505400    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-image-registry-private-configuration\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.505479 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505470    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-ca-trust-extracted\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.505617 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-bound-sa-token\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.505617 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-trusted-ca\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.505712 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505646    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.505712 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.505679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsm9c\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-kube-api-access-xsm9c\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.606664 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-installation-pull-secrets\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.606823 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96bk9\" (UniqueName: \"kubernetes.io/projected/5601a01b-0669-445d-8b56-3f843ec62a65-kube-api-access-96bk9\") pod \"ingress-canary-twkmz\" (UID: \"5601a01b-0669-445d-8b56-3f843ec62a65\") " pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.606823 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606708    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420f2a0e-48e0-4144-9118-d954a0ccd28a-tmp-dir\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.606823 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606736    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d7zk\" (UniqueName: \"kubernetes.io/projected/420f2a0e-48e0-4144-9118-d954a0ccd28a-kube-api-access-7d7zk\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.606823 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606805    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-certificates\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-image-registry-private-configuration\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606868    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-ca-trust-extracted\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-bound-sa-token\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/420f2a0e-48e0-4144-9118-d954a0ccd28a-config-volume\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606929    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-trusted-ca\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.606990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.607013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsm9c\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-kube-api-access-xsm9c\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607368 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.607033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5601a01b-0669-445d-8b56-3f843ec62a65-cert\") pod \"ingress-canary-twkmz\" (UID: \"5601a01b-0669-445d-8b56-3f843ec62a65\") " pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.607452 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:31.607432    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:37:31.607496 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:31.607456    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:37:31.607496 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.607464    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-ca-trust-extracted\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607562 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.607453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-certificates\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.607562 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:31.607522    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:37:32.107504368 +0000 UTC m=+36.572402418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:37:31.607941 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.607919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-trusted-ca\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.611785 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.611722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-image-registry-private-configuration\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.611866 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.611779    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-installation-pull-secrets\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.619781 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.619735    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsm9c\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-kube-api-access-xsm9c\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.620041 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.620021    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-bound-sa-token\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:31.645675 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.645639    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:31.708206 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/420f2a0e-48e0-4144-9118-d954a0ccd28a-config-volume\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.708313 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.708313 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708271    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5601a01b-0669-445d-8b56-3f843ec62a65-cert\") pod \"ingress-canary-twkmz\" (UID: \"5601a01b-0669-445d-8b56-3f843ec62a65\") " pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.708425 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708318    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96bk9\" (UniqueName: \"kubernetes.io/projected/5601a01b-0669-445d-8b56-3f843ec62a65-kube-api-access-96bk9\") pod \"ingress-canary-twkmz\" (UID: \"5601a01b-0669-445d-8b56-3f843ec62a65\") " pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.708480 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:31.708420    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:37:31.708480 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708439    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420f2a0e-48e0-4144-9118-d954a0ccd28a-tmp-dir\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.708577 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:31.708492    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:37:32.208468173 +0000 UTC m=+36.673366215 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:37:31.708577 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708515    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d7zk\" (UniqueName: \"kubernetes.io/projected/420f2a0e-48e0-4144-9118-d954a0ccd28a-kube-api-access-7d7zk\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.708737 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708716    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/420f2a0e-48e0-4144-9118-d954a0ccd28a-tmp-dir\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.708833 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.708813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/420f2a0e-48e0-4144-9118-d954a0ccd28a-config-volume\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.710957 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.710936    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5601a01b-0669-445d-8b56-3f843ec62a65-cert\") pod \"ingress-canary-twkmz\" (UID: \"5601a01b-0669-445d-8b56-3f843ec62a65\") " pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.719702 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.719679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d7zk\" (UniqueName: \"kubernetes.io/projected/420f2a0e-48e0-4144-9118-d954a0ccd28a-kube-api-access-7d7zk\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:31.722642 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.722617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96bk9\" (UniqueName: \"kubernetes.io/projected/5601a01b-0669-445d-8b56-3f843ec62a65-kube-api-access-96bk9\") pod \"ingress-canary-twkmz\" (UID: \"5601a01b-0669-445d-8b56-3f843ec62a65\") " pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.747442 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.747423    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-twkmz"
Jul 22 12:37:31.889938 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:31.889875    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-twkmz"]
Jul 22 12:37:31.893309 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:37:31.893282    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5601a01b_0669_445d_8b56_3f843ec62a65.slice/crio-49686b66b6dcfab8c564c542250384c8556ca7025894dcfd8eff2516273fee4f WatchSource:0}: Error finding container 49686b66b6dcfab8c564c542250384c8556ca7025894dcfd8eff2516273fee4f: Status 404 returned error can't find the container with id 49686b66b6dcfab8c564c542250384c8556ca7025894dcfd8eff2516273fee4f
Jul 22 12:37:32.012646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.012613    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:37:32.012646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.012633    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:37:32.012859 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.012836    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:37:32.015799 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.015775    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:37:32.015911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.015818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:37:32.015911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.015850    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5w6qj\""
Jul 22 12:37:32.015911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.015780    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:37:32.016093 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.016069    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j8xb2\""
Jul 22 12:37:32.016192 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.016131    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:37:32.111404 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.111380    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:32.111532 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:32.111518    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:37:32.111568 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:32.111535    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:37:32.111607 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:32.111593    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:37:33.11156691 +0000 UTC m=+37.576464940 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:37:32.165092 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.165032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-twkmz" event={"ID":"5601a01b-0669-445d-8b56-3f843ec62a65","Type":"ContainerStarted","Data":"49686b66b6dcfab8c564c542250384c8556ca7025894dcfd8eff2516273fee4f"}
Jul 22 12:37:32.212366 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.212344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:32.212536 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:32.212516    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:37:32.212600 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:32.212586    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:37:33.212567712 +0000 UTC m=+37.677465753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:37:32.419193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.419120    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:32.595170 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.595138    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:32.736864 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.736822    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:32.841256 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.841076    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:32.926813 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:32.926782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:33.022575 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:33.022463    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:33.119880 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:33.119849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:33.120042 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:33.120022    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:37:33.120042 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:33.120035    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:37:33.120139 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:33.120078    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:37:35.120063855 +0000 UTC m=+39.584961882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:37:33.220673 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:33.220635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:33.220895 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:33.220874    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:37:33.220973 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:33.220937    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:37:35.220918952 +0000 UTC m=+39.685816980 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:37:33.601135 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:33.601091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:33.893958 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:33.893874    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:33.994417 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:33.994387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:34.020139 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.020115    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"]
Jul 22 12:37:34.024616 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.024598    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.027663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.027637    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 22 12:37:34.028817 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.028796    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 22 12:37:34.028919 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.028814    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bmplt\""
Jul 22 12:37:34.028919 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.028872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4457d02c-7af3-4aed-80e4-75847674aa27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-686485844-6x67f\" (UID: \"4457d02c-7af3-4aed-80e4-75847674aa27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.028919 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.028905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gc5rc\" (UniqueName: \"kubernetes.io/projected/4457d02c-7af3-4aed-80e4-75847674aa27-kube-api-access-gc5rc\") pod \"managed-serviceaccount-addon-agent-686485844-6x67f\" (UID: \"4457d02c-7af3-4aed-80e4-75847674aa27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.029066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.029038    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 22 12:37:34.029098 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.029062    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 22 12:37:34.035663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.035636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"]
Jul 22 12:37:34.059275 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.059257    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"]
Jul 22 12:37:34.065823 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.065795    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"]
Jul 22 12:37:34.065932 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.065913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.068736 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.068719    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 22 12:37:34.071118 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.071099    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.073237 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.073219    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"]
Jul 22 12:37:34.074449 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.073581    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 22 12:37:34.074449 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.073588    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 22 12:37:34.074449 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.073919    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 22 12:37:34.074449 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.074170    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 22 12:37:34.083329 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.083257    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:34.083684 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.083664    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"]
Jul 22 12:37:34.129827 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.129804    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4457d02c-7af3-4aed-80e4-75847674aa27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-686485844-6x67f\" (UID: \"4457d02c-7af3-4aed-80e4-75847674aa27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.129942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.129840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-hub\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.129942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.129872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.129942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.129897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gc5rc\" (UniqueName: \"kubernetes.io/projected/4457d02c-7af3-4aed-80e4-75847674aa27-kube-api-access-gc5rc\") pod \"managed-serviceaccount-addon-agent-686485844-6x67f\" (UID: \"4457d02c-7af3-4aed-80e4-75847674aa27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.130185 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.130169    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab9e3fd-0238-448c-84f6-caebba4b214b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.130247 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.130197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2edc7573-66a1-4a32-9a04-5b9699ad8477-tmp\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.130247 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.130220    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2edc7573-66a1-4a32-9a04-5b9699ad8477-klusterlet-config\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.130356 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.130245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcv9k\" (UniqueName: \"kubernetes.io/projected/2edc7573-66a1-4a32-9a04-5b9699ad8477-kube-api-access-rcv9k\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.130356 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.130268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-ca\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.130482 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.130399    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.130482 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.130474    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26bvt\" (UniqueName: \"kubernetes.io/projected/2ab9e3fd-0238-448c-84f6-caebba4b214b-kube-api-access-26bvt\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.133244 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.133224    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4457d02c-7af3-4aed-80e4-75847674aa27-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-686485844-6x67f\" (UID: \"4457d02c-7af3-4aed-80e4-75847674aa27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.143665 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.143642    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gc5rc\" (UniqueName: \"kubernetes.io/projected/4457d02c-7af3-4aed-80e4-75847674aa27-kube-api-access-gc5rc\") pod \"managed-serviceaccount-addon-agent-686485844-6x67f\" (UID: \"4457d02c-7af3-4aed-80e4-75847674aa27\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.231227 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26bvt\" (UniqueName: \"kubernetes.io/projected/2ab9e3fd-0238-448c-84f6-caebba4b214b-kube-api-access-26bvt\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.231397 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-hub\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.231397 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231343    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.231397 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231368    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab9e3fd-0238-448c-84f6-caebba4b214b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.231397 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231390    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2edc7573-66a1-4a32-9a04-5b9699ad8477-tmp\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.231600 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2edc7573-66a1-4a32-9a04-5b9699ad8477-klusterlet-config\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.231600 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcv9k\" (UniqueName: \"kubernetes.io/projected/2edc7573-66a1-4a32-9a04-5b9699ad8477-kube-api-access-rcv9k\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.231600 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231461    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-ca\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.231600 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231507    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.231940 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.231915    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2edc7573-66a1-4a32-9a04-5b9699ad8477-tmp\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.232609 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.232585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/2ab9e3fd-0238-448c-84f6-caebba4b214b-additional-service-ca\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.234723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.234695    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.234988 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.234969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-ca\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.235114 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.235089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.235153 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.235140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/2ab9e3fd-0238-448c-84f6-caebba4b214b-hub\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.235190 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.235140    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/2edc7573-66a1-4a32-9a04-5b9699ad8477-klusterlet-config\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.241054 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.241031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26bvt\" (UniqueName: \"kubernetes.io/projected/2ab9e3fd-0238-448c-84f6-caebba4b214b-kube-api-access-26bvt\") pod \"cluster-proxy-proxy-agent-6dd9bbb654-2bj4t\" (UID: \"2ab9e3fd-0238-448c-84f6-caebba4b214b\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:34.248601 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.248576    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcv9k\" (UniqueName: \"kubernetes.io/projected/2edc7573-66a1-4a32-9a04-5b9699ad8477-kube-api-access-rcv9k\") pod \"klusterlet-addon-workmgr-54b957b88c-pzmrw\" (UID: \"2edc7573-66a1-4a32-9a04-5b9699ad8477\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.343643 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.343605    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"
Jul 22 12:37:34.378959 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.378924    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:34.385557 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:34.385533    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:37:35.140932 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:35.140895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:35.141371 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:35.141048    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:37:35.141371 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:35.141069    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:37:35.141371 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:35.141141    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:37:39.141117101 +0000 UTC m=+43.606015146 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:37:35.241385 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:35.241349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:35.241535 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:35.241499    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:37:35.241584 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:35.241559    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:37:39.241544895 +0000 UTC m=+43.706442952 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:37:35.598545 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:35.598328    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"]
Jul 22 12:37:35.602479 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:37:35.602455    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2edc7573_66a1_4a32_9a04_5b9699ad8477.slice/crio-f64d3333c57a744cadb49cfbde92394b0684da965f32e1ad3692b6b6c6657314 WatchSource:0}: Error finding container f64d3333c57a744cadb49cfbde92394b0684da965f32e1ad3692b6b6c6657314: Status 404 returned error can't find the container with id f64d3333c57a744cadb49cfbde92394b0684da965f32e1ad3692b6b6c6657314
Jul 22 12:37:35.610053 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:35.607898    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f"]
Jul 22 12:37:35.619313 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:35.618619    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"]
Jul 22 12:37:36.175252 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.175219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw" event={"ID":"2edc7573-66a1-4a32-9a04-5b9699ad8477","Type":"ContainerStarted","Data":"f64d3333c57a744cadb49cfbde92394b0684da965f32e1ad3692b6b6c6657314"}
Jul 22 12:37:36.177889 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.177863    2563 generic.go:356] "Generic (PLEG): container finished" podID="c57ae273-6130-41e0-9e27-04b2cb7334c0" containerID="e8c731388755d41a10208be64dbddba4f94bebb39e7261ba61a92842d9dc25dc" exitCode=0
Jul 22 12:37:36.178008 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.177940    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerDied","Data":"e8c731388755d41a10208be64dbddba4f94bebb39e7261ba61a92842d9dc25dc"}
Jul 22 12:37:36.179076 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.179057    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f" event={"ID":"4457d02c-7af3-4aed-80e4-75847674aa27","Type":"ContainerStarted","Data":"280c7104ea31e670284995e915f6e3224231390f2cc75878ecbca6894da207c9"}
Jul 22 12:37:36.180597 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.180574    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-twkmz" event={"ID":"5601a01b-0669-445d-8b56-3f843ec62a65","Type":"ContainerStarted","Data":"40afec7617edeaa63bfd08d0e4577ca4a0096402af4a291b9d4e5121b6161ce5"}
Jul 22 12:37:36.181714 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.181695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" event={"ID":"2ab9e3fd-0238-448c-84f6-caebba4b214b","Type":"ContainerStarted","Data":"94778dc6cc39a90cd53cc8fc382a8c9a934fb252579f6187d667a577663d2d86"}
Jul 22 12:37:36.216835 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.216788    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-twkmz" podStartSLOduration=1.701082466 podStartE2EDuration="5.216774609s" totalImagesPullingTime="3.515692143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:36.216163185 +0000 UTC m=+40.681061235" watchObservedRunningTime="2026-07-22 12:37:36.216774609 +0000 UTC m=+40.681672656"
Jul 22 12:37:36.857648 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:36.857599    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:37.189156 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:37.189071    2563 generic.go:356] "Generic (PLEG): container finished" podID="c57ae273-6130-41e0-9e27-04b2cb7334c0" containerID="fcbd4f22804dc85f3f673c25b2520be5c636a80ce3260823f3ffb0c88cc558f7" exitCode=0
Jul 22 12:37:37.190320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:37.190104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerDied","Data":"fcbd4f22804dc85f3f673c25b2520be5c636a80ce3260823f3ffb0c88cc558f7"}
Jul 22 12:37:37.202328 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:37.202248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:38.196609 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:38.196570    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" event={"ID":"c57ae273-6130-41e0-9e27-04b2cb7334c0","Type":"ContainerStarted","Data":"4377a4e0bd9504d7b24b4457e81b48893e7883d26345f9625949e24504cd6117"}
Jul 22 12:37:38.222244 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:38.222192    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4rhfq" podStartSLOduration=6.706505833 podStartE2EDuration="41.222173961s" totalImagesPullingTime="34.515668128s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:38.220699152 +0000 UTC m=+42.685597202" watchObservedRunningTime="2026-07-22 12:37:38.222173961 +0000 UTC m=+42.687072010"
Jul 22 12:37:38.392352 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:38.392281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:38.452838 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:38.452731    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:39.175383 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:39.175346    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:39.175563 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:39.175503    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:37:39.175563 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:39.175526    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:37:39.175677 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:39.175594    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:37:47.175574452 +0000 UTC m=+51.640472679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:37:39.276622 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:39.276583    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:39.277106 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:39.276714    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:37:39.277106 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:39.276805    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:37:47.276785633 +0000 UTC m=+51.741683667 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:37:41.200182 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:41.200142    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:41.301149 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:41.301069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log" limit=1024
Jul 22 12:37:43.212682 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.212646    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" event={"ID":"2ab9e3fd-0238-448c-84f6-caebba4b214b","Type":"ContainerStarted","Data":"06ecfd7d3009ff9db2e8770d0920939910c0365068054132a6116a2ae0bdef0a"}
Jul 22 12:37:43.213159 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.212698    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" event={"ID":"2ab9e3fd-0238-448c-84f6-caebba4b214b","Type":"ContainerStarted","Data":"50f87bbcc6a0d7f4007ee6cbba5dfc26031894108dc0d20261bfc8171591a0c0"}
Jul 22 12:37:43.213159 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.212713    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" event={"ID":"2ab9e3fd-0238-448c-84f6-caebba4b214b","Type":"ContainerStarted","Data":"9ca60b17f014e1a446fb2c8226cc495e8320a41e29507a4fa1cc2b65bf19ddcc"}
Jul 22 12:37:43.214135 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.214113    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw" event={"ID":"2edc7573-66a1-4a32-9a04-5b9699ad8477","Type":"ContainerStarted","Data":"769acd4e7d98bd25f58718da1abf6bfd881c2dd617d82ba373d53f3f52bae6e7"}
Jul 22 12:37:43.214232 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.214191    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:43.215647 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.215621    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f" event={"ID":"4457d02c-7af3-4aed-80e4-75847674aa27","Type":"ContainerStarted","Data":"13e20be4ec6615dcd40703795edf2735ea0dcb4c3742d6ff70e3dc484fdb7c27"}
Jul 22 12:37:43.216064 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.216046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw"
Jul 22 12:37:43.234533 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.234496    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" podStartSLOduration=2.63439144 podStartE2EDuration="9.23448213s" totalImagesPullingTime="6.60009069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:43.233355896 +0000 UTC m=+47.698253945" watchObservedRunningTime="2026-07-22 12:37:43.23448213 +0000 UTC m=+47.699380176"
Jul 22 12:37:43.251214 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.251177    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54b957b88c-pzmrw" podStartSLOduration=2.108448674 podStartE2EDuration="9.251165288s" totalImagesPullingTime="7.142716614s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:43.24977876 +0000 UTC m=+47.714676805" watchObservedRunningTime="2026-07-22 12:37:43.251165288 +0000 UTC m=+47.716063337"
Jul 22 12:37:43.271838 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:43.271800    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-686485844-6x67f" podStartSLOduration=2.666727814 podStartE2EDuration="9.271789007s" totalImagesPullingTime="6.605061193s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:37:43.271387992 +0000 UTC m=+47.736286053" watchObservedRunningTime="2026-07-22 12:37:43.271789007 +0000 UTC m=+47.736687055"
Jul 22 12:37:47.242271 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:47.242242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:37:47.242684 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:47.242392    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:37:47.242684 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:47.242411    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:37:47.242684 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:47.242461    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:38:03.242446552 +0000 UTC m=+67.707344579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:37:47.342558 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:47.342530    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:37:47.342903 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:47.342690    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:37:47.342903 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:37:47.342793    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:38:03.342768809 +0000 UTC m=+67.807666851 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:37:58.167654 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:37:58.167628    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6m94m"
Jul 22 12:38:01.746806 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.746771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:38:01.747186 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.746815    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:38:01.750162 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.750145    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 22 12:38:01.750232 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.750195    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 22 12:38:01.757314 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:01.757291    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 22 12:38:01.757457 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:01.757401    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs podName:7172bfdb-6e7f-4586-8319-c3e9990fa020 nodeName:}" failed. No retries permitted until 2026-07-22 12:39:05.757378377 +0000 UTC m=+130.222276419 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs") pod "network-metrics-daemon-546pc" (UID: "7172bfdb-6e7f-4586-8319-c3e9990fa020") : secret "metrics-daemon-secret" not found
Jul 22 12:38:01.760706 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.760683    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9185d527-afa2-47e7-ac1b-73aae2dcbc9c-original-pull-secret\") pod \"global-pull-secret-syncer-zfw59\" (UID: \"9185d527-afa2-47e7-ac1b-73aae2dcbc9c\") " pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:38:01.847107 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.847074    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:38:01.850348 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.850329    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 22 12:38:01.860036 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.860020    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 22 12:38:01.870413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:01.870393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfvcq\" (UniqueName: \"kubernetes.io/projected/1ecb27d5-2560-44dc-8ce0-992c6aa48e51-kube-api-access-cfvcq\") pod \"network-check-target-7sb8v\" (UID: \"1ecb27d5-2560-44dc-8ce0-992c6aa48e51\") " pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:38:02.024586 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:02.024530    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zfw59"
Jul 22 12:38:02.034998 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:02.034977    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-5w6qj\""
Jul 22 12:38:02.042253 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:02.042230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:38:02.161317 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:02.161290    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zfw59"]
Jul 22 12:38:02.164657 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:38:02.164634    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9185d527_afa2_47e7_ac1b_73aae2dcbc9c.slice/crio-883d7ed4929521fecc3a17f85f98f7d46546c26f310d6637ef94fce4440aabba WatchSource:0}: Error finding container 883d7ed4929521fecc3a17f85f98f7d46546c26f310d6637ef94fce4440aabba: Status 404 returned error can't find the container with id 883d7ed4929521fecc3a17f85f98f7d46546c26f310d6637ef94fce4440aabba
Jul 22 12:38:02.179418 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:02.179398    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-7sb8v"]
Jul 22 12:38:02.182097 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:38:02.182072    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ecb27d5_2560_44dc_8ce0_992c6aa48e51.slice/crio-b2b179fe88a85035b72b6c158ef88d6a5d3d90a60b11b1cac22796aea87af644 WatchSource:0}: Error finding container b2b179fe88a85035b72b6c158ef88d6a5d3d90a60b11b1cac22796aea87af644: Status 404 returned error can't find the container with id b2b179fe88a85035b72b6c158ef88d6a5d3d90a60b11b1cac22796aea87af644
Jul 22 12:38:02.274926 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:02.274865    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7sb8v" event={"ID":"1ecb27d5-2560-44dc-8ce0-992c6aa48e51","Type":"ContainerStarted","Data":"b2b179fe88a85035b72b6c158ef88d6a5d3d90a60b11b1cac22796aea87af644"}
Jul 22 12:38:02.275811 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:02.275792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zfw59" event={"ID":"9185d527-afa2-47e7-ac1b-73aae2dcbc9c","Type":"ContainerStarted","Data":"883d7ed4929521fecc3a17f85f98f7d46546c26f310d6637ef94fce4440aabba"}
Jul 22 12:38:03.256240 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:03.256204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:38:03.256718 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:03.256325    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:38:03.256718 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:03.256337    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:38:03.256718 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:03.256395    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:38:35.256374187 +0000 UTC m=+99.721272233 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:38:03.357582 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:03.357542    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:38:03.357814 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:03.357705    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:38:03.357814 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:03.357788    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:38:35.357773102 +0000 UTC m=+99.822671147 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:38:08.295908 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:08.295873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-7sb8v" event={"ID":"1ecb27d5-2560-44dc-8ce0-992c6aa48e51","Type":"ContainerStarted","Data":"78a38d2c76e2c9798a9270de92748ec380267b53aa73f55372305f0ba836a3e9"}
Jul 22 12:38:08.296395 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:08.296017    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:38:08.297303 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:08.297281    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zfw59" event={"ID":"9185d527-afa2-47e7-ac1b-73aae2dcbc9c","Type":"ContainerStarted","Data":"f07aeedf322b8fc59db26bc01514964d9e7969deaaa37e9d4db1c886ec544b24"}
Jul 22 12:38:08.313616 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:08.313575    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-7sb8v" podStartSLOduration=67.080977463 podStartE2EDuration="1m12.31355546s" totalImagesPullingTime="5.232577997s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:36:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:38:08.313115909 +0000 UTC m=+72.778013953" watchObservedRunningTime="2026-07-22 12:38:08.31355546 +0000 UTC m=+72.778453503"
Jul 22 12:38:08.329169 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:08.329127    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zfw59" podStartSLOduration=66.076373694 podStartE2EDuration="1m11.329115514s" totalImagesPullingTime="5.25274182s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:38:08.328204897 +0000 UTC m=+72.793102948" watchObservedRunningTime="2026-07-22 12:38:08.329115514 +0000 UTC m=+72.794013563"
Jul 22 12:38:35.284510 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:35.284473    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") pod \"image-registry-d7cdd955f-l8kmv\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") " pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:38:35.284996 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:35.284594    2563 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 22 12:38:35.284996 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:35.284606    2563 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-d7cdd955f-l8kmv: secret "image-registry-tls" not found
Jul 22 12:38:35.284996 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:35.284666    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls podName:872596ff-eb00-4c5d-bcd2-0cadbb500a1b nodeName:}" failed. No retries permitted until 2026-07-22 12:39:39.284650533 +0000 UTC m=+163.749548567 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls") pod "image-registry-d7cdd955f-l8kmv" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b") : secret "image-registry-tls" not found
Jul 22 12:38:35.385025 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:35.384987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:38:35.385197 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:35.385114    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 22 12:38:35.385197 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:35.385185    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls podName:420f2a0e-48e0-4144-9118-d954a0ccd28a nodeName:}" failed. No retries permitted until 2026-07-22 12:39:39.385167427 +0000 UTC m=+163.850065468 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls") pod "dns-default-spfbs" (UID: "420f2a0e-48e0-4144-9118-d954a0ccd28a") : secret "dns-default-metrics-tls" not found
Jul 22 12:38:39.303484 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:39.303452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-7sb8v"
Jul 22 12:38:46.582249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.582214    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"]
Jul 22 12:38:46.584066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.584046    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"
Jul 22 12:38:46.587457 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.587436    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-8dvhh\""
Jul 22 12:38:46.594886 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.594863    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"]
Jul 22 12:38:46.661417 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.661384    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9ftp\" (UniqueName: \"kubernetes.io/projected/0410ef2f-5dff-4df5-8583-888b3ba2d6cf-kube-api-access-z9ftp\") pod \"network-check-source-7bc5b6bf85-9hcnc\" (UID: \"0410ef2f-5dff-4df5-8583-888b3ba2d6cf\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"
Jul 22 12:38:46.762596 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.762571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9ftp\" (UniqueName: \"kubernetes.io/projected/0410ef2f-5dff-4df5-8583-888b3ba2d6cf-kube-api-access-z9ftp\") pod \"network-check-source-7bc5b6bf85-9hcnc\" (UID: \"0410ef2f-5dff-4df5-8583-888b3ba2d6cf\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"
Jul 22 12:38:46.771593 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.771573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9ftp\" (UniqueName: \"kubernetes.io/projected/0410ef2f-5dff-4df5-8583-888b3ba2d6cf-kube-api-access-z9ftp\") pod \"network-check-source-7bc5b6bf85-9hcnc\" (UID: \"0410ef2f-5dff-4df5-8583-888b3ba2d6cf\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"
Jul 22 12:38:46.892944 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:46.892877    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"
Jul 22 12:38:47.016063 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:47.016036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc"]
Jul 22 12:38:47.018252 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:38:47.018224    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0410ef2f_5dff_4df5_8583_888b3ba2d6cf.slice/crio-7dbb58c7ccde76e59d2602a30d878e25805143dee3f9068763db55e4d8a69e2c WatchSource:0}: Error finding container 7dbb58c7ccde76e59d2602a30d878e25805143dee3f9068763db55e4d8a69e2c: Status 404 returned error can't find the container with id 7dbb58c7ccde76e59d2602a30d878e25805143dee3f9068763db55e4d8a69e2c
Jul 22 12:38:47.409015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:47.408983    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc" event={"ID":"0410ef2f-5dff-4df5-8583-888b3ba2d6cf","Type":"ContainerStarted","Data":"4d56d4586706eb121a792c2f1eedee7e89d83f210edebff3a0523e2f7a5bbb59"}
Jul 22 12:38:47.409182 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:47.409056    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc" event={"ID":"0410ef2f-5dff-4df5-8583-888b3ba2d6cf","Type":"ContainerStarted","Data":"7dbb58c7ccde76e59d2602a30d878e25805143dee3f9068763db55e4d8a69e2c"}
Jul 22 12:38:47.425699 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:47.425650    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-9hcnc" podStartSLOduration=1.425636984 podStartE2EDuration="1.425636984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:38:47.425427536 +0000 UTC m=+111.890325586" watchObservedRunningTime="2026-07-22 12:38:47.425636984 +0000 UTC m=+111.890535011"
Jul 22 12:38:50.715760 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.715708    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5s6bp"]
Jul 22 12:38:50.717881 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.717859    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.730567 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.730550    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 22 12:38:50.730784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.730767    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 22 12:38:50.730850 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.730786    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmzhg\""
Jul 22 12:38:50.731010 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.730998    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 22 12:38:50.731193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.731177    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 22 12:38:50.741000 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.740980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5s6bp"]
Jul 22 12:38:50.791871 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.791845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.791972 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.791898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9507753-cea0-4d7b-839d-5da83cd24d4d-crio-socket\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.791972 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.791921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.792040 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.791980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9507753-cea0-4d7b-839d-5da83cd24d4d-data-volume\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.792040 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.792018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qhx2\" (UniqueName: \"kubernetes.io/projected/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-api-access-5qhx2\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.892510 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.892467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.892698 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.892556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9507753-cea0-4d7b-839d-5da83cd24d4d-crio-socket\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.892698 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:50.892587    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:38:50.892698 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.892594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.892698 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:50.892653    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:51.392637582 +0000 UTC m=+115.857535644 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : configmap "kube-rbac-proxy" not found
Jul 22 12:38:50.892698 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.892680    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d9507753-cea0-4d7b-839d-5da83cd24d4d-crio-socket\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.892987 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.892682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9507753-cea0-4d7b-839d-5da83cd24d4d-data-volume\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.892987 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.892734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qhx2\" (UniqueName: \"kubernetes.io/projected/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-api-access-5qhx2\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.892987 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:50.892685    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:50.892987 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:50.892841    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:51.392822123 +0000 UTC m=+115.857720150 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:50.892987 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.892977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9507753-cea0-4d7b-839d-5da83cd24d4d-data-volume\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:50.902045 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:50.902027    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qhx2\" (UniqueName: \"kubernetes.io/projected/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-api-access-5qhx2\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:51.397162 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:51.397123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:51.397310 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:51.397207    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:51.397310 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:51.397257    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:38:51.397401 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:51.397322    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:51.397401 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:51.397328    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:52.397310054 +0000 UTC m=+116.862208104 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : configmap "kube-rbac-proxy" not found
Jul 22 12:38:51.397401 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:51.397356    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:52.397346932 +0000 UTC m=+116.862244972 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:52.404045 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:52.404011    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:52.404485 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:52.404076    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:52.404485 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:52.404148    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:38:52.404485 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:52.404159    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:52.404485 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:52.404209    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:54.404194248 +0000 UTC m=+118.869092274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : configmap "kube-rbac-proxy" not found
Jul 22 12:38:52.404485 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:52.404225    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:54.404217089 +0000 UTC m=+118.869115116 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:52.478843 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:52.478823    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7cb9k_847d1d73-3c3c-4cce-a5ce-2f05752884f5/dns-node-resolver/0.log"
Jul 22 12:38:53.880515 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:53.880485    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xvbww_dba8b7c7-8dd2-4913-ab86-8e65582cb93f/node-ca/0.log"
Jul 22 12:38:54.420004 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:54.419976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:54.420161 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:54.420033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:54.420161 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:54.420109    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:38:54.420161 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:54.420127    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:54.420161 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:54.420160    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:58.420147122 +0000 UTC m=+122.885045149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : configmap "kube-rbac-proxy" not found
Jul 22 12:38:54.420296 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:54.420193    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:38:58.420180557 +0000 UTC m=+122.885078583 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:54.481156 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:54.481125    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-twkmz_5601a01b-0669-445d-8b56-3f843ec62a65/serve-healthcheck-canary/0.log"
Jul 22 12:38:58.447792 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:58.447761    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:58.448164 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:38:58.447826    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:38:58.448164 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:58.447885    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:58.448164 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:58.447891    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 22 12:38:58.448164 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:58.447951    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:39:06.447937072 +0000 UTC m=+130.912835099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:38:58.448164 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:38:58.447966    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:39:06.447959659 +0000 UTC m=+130.912857686 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : configmap "kube-rbac-proxy" not found
Jul 22 12:39:04.386917 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:04.386871    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" podUID="2ab9e3fd-0238-448c-84f6-caebba4b214b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 12:39:05.802804 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:05.802741    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:39:05.805379 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:05.805353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7172bfdb-6e7f-4586-8319-c3e9990fa020-metrics-certs\") pod \"network-metrics-daemon-546pc\" (UID: \"7172bfdb-6e7f-4586-8319-c3e9990fa020\") " pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:39:05.940469 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:05.940439    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j8xb2\""
Jul 22 12:39:05.948362 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:05.948345    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-546pc"
Jul 22 12:39:06.070814 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:06.070735    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-546pc"]
Jul 22 12:39:06.073509 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:39:06.073489    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7172bfdb_6e7f_4586_8319_c3e9990fa020.slice/crio-b4e62bfdceed9caf58a489bf048e85f9a56ccb9fd5d6fa5cff3a7e32ba69bc19 WatchSource:0}: Error finding container b4e62bfdceed9caf58a489bf048e85f9a56ccb9fd5d6fa5cff3a7e32ba69bc19: Status 404 returned error can't find the container with id b4e62bfdceed9caf58a489bf048e85f9a56ccb9fd5d6fa5cff3a7e32ba69bc19
Jul 22 12:39:06.464320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:06.464293    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-546pc" event={"ID":"7172bfdb-6e7f-4586-8319-c3e9990fa020","Type":"ContainerStarted","Data":"b4e62bfdceed9caf58a489bf048e85f9a56ccb9fd5d6fa5cff3a7e32ba69bc19"}
Jul 22 12:39:06.508805 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:06.508769    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:39:06.508923 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:06.508834    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:39:06.508965 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:06.508924    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 22 12:39:06.509016 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:06.509005    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls podName:d9507753-cea0-4d7b-839d-5da83cd24d4d nodeName:}" failed. No retries permitted until 2026-07-22 12:39:22.508986909 +0000 UTC m=+146.973884936 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5s6bp" (UID: "d9507753-cea0-4d7b-839d-5da83cd24d4d") : secret "insights-runtime-extractor-tls" not found
Jul 22 12:39:06.509484 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:06.509465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d9507753-cea0-4d7b-839d-5da83cd24d4d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:39:07.469738 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:07.469705    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-546pc" event={"ID":"7172bfdb-6e7f-4586-8319-c3e9990fa020","Type":"ContainerStarted","Data":"2ddc756674f2cac1fa1588e8aa4c102cb259efc62bc262c3f3a432fc37630831"}
Jul 22 12:39:07.470195 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:07.469793    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-546pc" event={"ID":"7172bfdb-6e7f-4586-8319-c3e9990fa020","Type":"ContainerStarted","Data":"5c315fcf19fa320a32b86a4133e93e8e2dbe4df2d9757720b07a097ccd74a7fb"}
Jul 22 12:39:07.490856 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:07.490808    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-546pc" podStartSLOduration=129.47477589 podStartE2EDuration="2m10.490793049s" totalImagesPullingTime="1.016017159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:36:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:39:07.488937403 +0000 UTC m=+131.953835452" watchObservedRunningTime="2026-07-22 12:39:07.490793049 +0000 UTC m=+131.955691098"
Jul 22 12:39:14.386504 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:14.386464    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" podUID="2ab9e3fd-0238-448c-84f6-caebba4b214b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 12:39:22.532778 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:22.531778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:39:22.535011 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:22.534987    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d9507753-cea0-4d7b-839d-5da83cd24d4d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5s6bp\" (UID: \"d9507753-cea0-4d7b-839d-5da83cd24d4d\") " pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:39:22.829648 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:22.829576    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-dmzhg\""
Jul 22 12:39:22.836902 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:22.836874    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5s6bp"
Jul 22 12:39:22.965152 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:22.965036    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5s6bp"]
Jul 22 12:39:22.968463 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:39:22.968418    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9507753_cea0_4d7b_839d_5da83cd24d4d.slice/crio-2cf32f89fc1c7682096457569b7e8b3b9baaa2f9c7a91a88b9b79228a66e7bb3 WatchSource:0}: Error finding container 2cf32f89fc1c7682096457569b7e8b3b9baaa2f9c7a91a88b9b79228a66e7bb3: Status 404 returned error can't find the container with id 2cf32f89fc1c7682096457569b7e8b3b9baaa2f9c7a91a88b9b79228a66e7bb3
Jul 22 12:39:23.519092 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:23.519007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5s6bp" event={"ID":"d9507753-cea0-4d7b-839d-5da83cd24d4d","Type":"ContainerStarted","Data":"2cf32f89fc1c7682096457569b7e8b3b9baaa2f9c7a91a88b9b79228a66e7bb3"}
Jul 22 12:39:24.387338 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:24.387296    2563 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" podUID="2ab9e3fd-0238-448c-84f6-caebba4b214b" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 22 12:39:24.387734 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:24.387394    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t"
Jul 22 12:39:24.388101 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:24.388059    2563 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"06ecfd7d3009ff9db2e8770d0920939910c0365068054132a6116a2ae0bdef0a"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 22 12:39:24.388163 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:24.388132    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" podUID="2ab9e3fd-0238-448c-84f6-caebba4b214b" containerName="service-proxy" containerID="cri-o://06ecfd7d3009ff9db2e8770d0920939910c0365068054132a6116a2ae0bdef0a" gracePeriod=30
Jul 22 12:39:24.524393 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:24.524365    2563 generic.go:356] "Generic (PLEG): container finished" podID="2ab9e3fd-0238-448c-84f6-caebba4b214b" containerID="06ecfd7d3009ff9db2e8770d0920939910c0365068054132a6116a2ae0bdef0a" exitCode=2
Jul 22 12:39:24.524528 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:24.524419    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" event={"ID":"2ab9e3fd-0238-448c-84f6-caebba4b214b","Type":"ContainerDied","Data":"06ecfd7d3009ff9db2e8770d0920939910c0365068054132a6116a2ae0bdef0a"}
Jul 22 12:39:25.531170 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:25.531125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-6dd9bbb654-2bj4t" event={"ID":"2ab9e3fd-0238-448c-84f6-caebba4b214b","Type":"ContainerStarted","Data":"742976cc95f61df66a3f912296efb3c11e1177fcdb71a03e4fcacd21c48a371a"}
Jul 22 12:39:28.542335 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:28.542302    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5s6bp" event={"ID":"d9507753-cea0-4d7b-839d-5da83cd24d4d","Type":"ContainerStarted","Data":"82aa0f22798bfe00cf2160fc4dfda28c63cd8ea4aee5bd03273f3e01b571bf4e"}
Jul 22 12:39:32.558868 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:32.558831    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5s6bp" event={"ID":"d9507753-cea0-4d7b-839d-5da83cd24d4d","Type":"ContainerStarted","Data":"cd86ef5685fffa2a09f3aae1bff11ccfc548cd2bca4f5faa9e57c33abc582e32"}
Jul 22 12:39:34.414379 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.414345    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lf8hv"]
Jul 22 12:39:34.418179 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:34.418150    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv" podUID="872596ff-eb00-4c5d-bcd2-0cadbb500a1b"
Jul 22 12:39:34.428526 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.428506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.431362 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.431340    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 22 12:39:34.431491 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.431471    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 22 12:39:34.431576 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.431539    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 22 12:39:34.431637 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.431579    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 22 12:39:34.432478 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.432459    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 22 12:39:34.432478 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.432468    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 22 12:39:34.432635 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.432503    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-vqfsx\""
Jul 22 12:39:34.441634 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:34.441601    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-spfbs" podUID="420f2a0e-48e0-4144-9118-d954a0ccd28a"
Jul 22 12:39:34.527542 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5hfz\" (UniqueName: \"kubernetes.io/projected/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-kube-api-access-f5hfz\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.527542 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527533    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.527819 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527615    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-root\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.527819 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-sys\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.527819 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527690    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-textfile\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.527819 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527710    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-wtmp\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.527819 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527791    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.528011 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527853    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-metrics-client-ca\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.528011 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.527912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.566490 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.566411    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:39:34.566490 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.566458    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-spfbs"
Jul 22 12:39:34.628905 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.628872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-sys\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.628905 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.628906    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-textfile\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629148 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.628931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-wtmp\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629148 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.628972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-sys\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629148 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629148 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-metrics-client-ca\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629154    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-wtmp\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629258    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5hfz\" (UniqueName: \"kubernetes.io/projected/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-kube-api-access-f5hfz\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629320 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-textfile\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629549 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629480    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-root\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629643 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629565    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-root\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629643 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629600    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-metrics-client-ca\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.629643 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:34.629611    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 12:39:34.629834 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:34.629673    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls podName:61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f nodeName:}" failed. No retries permitted until 2026-07-22 12:39:35.129656219 +0000 UTC m=+159.594554270 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls") pod "node-exporter-lf8hv" (UID: "61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f") : secret "node-exporter-tls" not found
Jul 22 12:39:34.629834 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.629686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.632374 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.632352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:34.638838 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:34.638813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5hfz\" (UniqueName: \"kubernetes.io/projected/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-kube-api-access-f5hfz\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:35.134533 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:35.134496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:35.134708 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:35.134678    2563 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 22 12:39:35.134779 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:35.134772    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls podName:61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f nodeName:}" failed. No retries permitted until 2026-07-22 12:39:36.134735672 +0000 UTC m=+160.599633699 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls") pod "node-exporter-lf8hv" (UID: "61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f") : secret "node-exporter-tls" not found
Jul 22 12:39:36.142289 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.142251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:36.144682 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.144655    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f-node-exporter-tls\") pod \"node-exporter-lf8hv\" (UID: \"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f\") " pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:36.239834 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.239797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lf8hv"
Jul 22 12:39:36.250408 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:39:36.250380    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61d2b0d5_a1dd_4475_b642_a3f2f62b1d8f.slice/crio-4a7f799924382afebb94a68d0f59c94ae8307a1f48739dd9a2a8e5f4b89a9283 WatchSource:0}: Error finding container 4a7f799924382afebb94a68d0f59c94ae8307a1f48739dd9a2a8e5f4b89a9283: Status 404 returned error can't find the container with id 4a7f799924382afebb94a68d0f59c94ae8307a1f48739dd9a2a8e5f4b89a9283
Jul 22 12:39:36.417346 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.417261    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d7cdd955f-l8kmv"]
Jul 22 12:39:36.417495 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:39:36.417478    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv" podUID="872596ff-eb00-4c5d-bcd2-0cadbb500a1b"
Jul 22 12:39:36.573445 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.573407    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lf8hv" event={"ID":"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f","Type":"ContainerStarted","Data":"4a7f799924382afebb94a68d0f59c94ae8307a1f48739dd9a2a8e5f4b89a9283"}
Jul 22 12:39:36.575369 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.575349    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:39:36.575505 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.575345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5s6bp" event={"ID":"d9507753-cea0-4d7b-839d-5da83cd24d4d","Type":"ContainerStarted","Data":"1419ddd72312a31bd667e265eb0533bf743cc6ad2dcc79f5ea96cdef52dc2ac8"}
Jul 22 12:39:36.582865 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.582847    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:39:36.621619 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.621578    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5s6bp" podStartSLOduration=34.19576979 podStartE2EDuration="46.621562851s" totalImagesPullingTime="12.425793061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 12:38:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:39:36.620101155 +0000 UTC m=+161.084999204" watchObservedRunningTime="2026-07-22 12:39:36.621562851 +0000 UTC m=+161.086460900"
Jul 22 12:39:36.747085 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-installation-pull-secrets\") pod \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") "
Jul 22 12:39:36.747249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747124    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-ca-trust-extracted\") pod \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") "
Jul 22 12:39:36.747249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747152    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-certificates\") pod \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") "
Jul 22 12:39:36.747249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747199    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-bound-sa-token\") pod \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") "
Jul 22 12:39:36.747249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747228    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-image-registry-private-configuration\") pod \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") "
Jul 22 12:39:36.747468 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747273    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-trusted-ca\") pod \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") "
Jul 22 12:39:36.747468 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747299    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsm9c\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-kube-api-access-xsm9c\") pod \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\" (UID: \"872596ff-eb00-4c5d-bcd2-0cadbb500a1b\") "
Jul 22 12:39:36.747564 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747465    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "872596ff-eb00-4c5d-bcd2-0cadbb500a1b" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:39:36.747615 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747589    2563 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-ca-trust-extracted\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:36.747846 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.747822    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "872596ff-eb00-4c5d-bcd2-0cadbb500a1b" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:39:36.748027 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.748002    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "872596ff-eb00-4c5d-bcd2-0cadbb500a1b" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:39:36.750026 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.749979    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "872596ff-eb00-4c5d-bcd2-0cadbb500a1b" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:39:36.750463 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.750440    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-kube-api-access-xsm9c" (OuterVolumeSpecName: "kube-api-access-xsm9c") pod "872596ff-eb00-4c5d-bcd2-0cadbb500a1b" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b"). InnerVolumeSpecName "kube-api-access-xsm9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:36.750561 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.750484    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "872596ff-eb00-4c5d-bcd2-0cadbb500a1b" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:39:36.750561 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.750513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "872596ff-eb00-4c5d-bcd2-0cadbb500a1b" (UID: "872596ff-eb00-4c5d-bcd2-0cadbb500a1b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:39:36.848822 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.848790    2563 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-certificates\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:36.848822 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.848823    2563 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-bound-sa-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:36.849049 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.848839    2563 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-image-registry-private-configuration\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:36.849049 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.848853    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-trusted-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:36.849049 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.848867    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsm9c\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-kube-api-access-xsm9c\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:36.849049 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:36.848879    2563 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-installation-pull-secrets\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:37.580437 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:37.580351    2563 generic.go:356] "Generic (PLEG): container finished" podID="61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f" containerID="d922262e49ee983aeee607d6baad1bca64129155e3b1fa8c4cb1851811387aad" exitCode=0
Jul 22 12:39:37.580859 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:37.580448    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lf8hv" event={"ID":"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f","Type":"ContainerDied","Data":"d922262e49ee983aeee607d6baad1bca64129155e3b1fa8c4cb1851811387aad"}
Jul 22 12:39:37.580859 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:37.580813    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-d7cdd955f-l8kmv"
Jul 22 12:39:37.626070 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:37.626043    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-d7cdd955f-l8kmv"]
Jul 22 12:39:37.630276 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:37.630252    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-d7cdd955f-l8kmv"]
Jul 22 12:39:37.756989 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:37.756705    2563 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872596ff-eb00-4c5d-bcd2-0cadbb500a1b-registry-tls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:39:38.016616 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:38.016582    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="872596ff-eb00-4c5d-bcd2-0cadbb500a1b" path="/var/lib/kubelet/pods/872596ff-eb00-4c5d-bcd2-0cadbb500a1b/volumes"
Jul 22 12:39:38.586042 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:38.586007    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lf8hv" event={"ID":"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f","Type":"ContainerStarted","Data":"6fcf54c25f6f5f6ab6c1519cf00e9d0a82588531908e8cf7c0f0fd5147cb31b4"}
Jul 22 12:39:38.586416 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:38.586067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lf8hv" event={"ID":"61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f","Type":"ContainerStarted","Data":"52c15eb09e09d9903c616f108fd4949dc9093749e7ff268deb2aae7446dde84d"}
Jul 22 12:39:38.608882 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:38.608835    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lf8hv" podStartSLOduration=3.582297468 podStartE2EDuration="4.608821767s" totalImagesPullingTime="956.514823ms" totalInitContainerRuntime="70.009476ms" isStatefulPod=true podCreationTimestamp="2026-07-22 12:39:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:39:38.607550108 +0000 UTC m=+163.072448168" watchObservedRunningTime="2026-07-22 12:39:38.608821767 +0000 UTC m=+163.073719816"
Jul 22 12:39:39.472110 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:39.472079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:39:39.474410 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:39.474390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/420f2a0e-48e0-4144-9118-d954a0ccd28a-metrics-tls\") pod \"dns-default-spfbs\" (UID: \"420f2a0e-48e0-4144-9118-d954a0ccd28a\") " pod="openshift-dns/dns-default-spfbs"
Jul 22 12:39:39.670341 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:39.670308    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ntb22\""
Jul 22 12:39:39.678766 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:39.678727    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-spfbs"
Jul 22 12:39:39.820093 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:39.820068    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-spfbs"]
Jul 22 12:39:39.822455 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:39:39.822427    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod420f2a0e_48e0_4144_9118_d954a0ccd28a.slice/crio-7179818d2e94999feb389c89d5dc3cd1c29d18118a303236b9eba6cac738b76e WatchSource:0}: Error finding container 7179818d2e94999feb389c89d5dc3cd1c29d18118a303236b9eba6cac738b76e: Status 404 returned error can't find the container with id 7179818d2e94999feb389c89d5dc3cd1c29d18118a303236b9eba6cac738b76e
Jul 22 12:39:40.595057 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:40.595018    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-spfbs" event={"ID":"420f2a0e-48e0-4144-9118-d954a0ccd28a","Type":"ContainerStarted","Data":"7179818d2e94999feb389c89d5dc3cd1c29d18118a303236b9eba6cac738b76e"}
Jul 22 12:39:42.603006 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:42.602961    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-spfbs" event={"ID":"420f2a0e-48e0-4144-9118-d954a0ccd28a","Type":"ContainerStarted","Data":"f4e7756a594ddf7884b96e4fc34ba13f4f594b8969d7e99bd4337d3ebff43419"}
Jul 22 12:39:42.603492 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:42.603015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-spfbs" event={"ID":"420f2a0e-48e0-4144-9118-d954a0ccd28a","Type":"ContainerStarted","Data":"2d94a851821ad55d87d3029ac62bb4bbfe4ad524fabfe7653fd688e4fe6a7efe"}
Jul 22 12:39:42.603492 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:42.603102    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-spfbs"
Jul 22 12:39:42.625396 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:42.625350    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-spfbs" podStartSLOduration=129.85484487 podStartE2EDuration="2m11.625335361s" totalImagesPullingTime="1.770490491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:37:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:39:42.624608623 +0000 UTC m=+167.089506672" watchObservedRunningTime="2026-07-22 12:39:42.625335361 +0000 UTC m=+167.090233410"
Jul 22 12:39:52.608660 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:39:52.608563    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-spfbs"
Jul 22 12:41:55.918587 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:41:55.918562    2563 kubelet.go:1648] "Image garbage collection succeeded"
Jul 22 12:43:54.331056 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.331024    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"]
Jul 22 12:43:54.333934 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.333918    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.336603 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.336582    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 22 12:43:54.337817 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.337799    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 22 12:43:54.337874 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.337834    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-nm6nj\""
Jul 22 12:43:54.342147 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.342124    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"]
Jul 22 12:43:54.425875 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.425847    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wglk\" (UniqueName: \"kubernetes.io/projected/41cc80f4-6019-4efd-8352-7b29cfe4d4f8-kube-api-access-2wglk\") pod \"openshift-lws-operator-fd8ccff4c-587h5\" (UID: \"41cc80f4-6019-4efd-8352-7b29cfe4d4f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.426045 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.426025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41cc80f4-6019-4efd-8352-7b29cfe4d4f8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-587h5\" (UID: \"41cc80f4-6019-4efd-8352-7b29cfe4d4f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.527018 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.526980    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wglk\" (UniqueName: \"kubernetes.io/projected/41cc80f4-6019-4efd-8352-7b29cfe4d4f8-kube-api-access-2wglk\") pod \"openshift-lws-operator-fd8ccff4c-587h5\" (UID: \"41cc80f4-6019-4efd-8352-7b29cfe4d4f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.527134 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.527033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41cc80f4-6019-4efd-8352-7b29cfe4d4f8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-587h5\" (UID: \"41cc80f4-6019-4efd-8352-7b29cfe4d4f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.527419 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.527399    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41cc80f4-6019-4efd-8352-7b29cfe4d4f8-tmp\") pod \"openshift-lws-operator-fd8ccff4c-587h5\" (UID: \"41cc80f4-6019-4efd-8352-7b29cfe4d4f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.536218 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.536187    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wglk\" (UniqueName: \"kubernetes.io/projected/41cc80f4-6019-4efd-8352-7b29cfe4d4f8-kube-api-access-2wglk\") pod \"openshift-lws-operator-fd8ccff4c-587h5\" (UID: \"41cc80f4-6019-4efd-8352-7b29cfe4d4f8\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.642326 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.642273    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"
Jul 22 12:43:54.764396 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.764370    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5"]
Jul 22 12:43:54.767346 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:43:54.767318    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41cc80f4_6019_4efd_8352_7b29cfe4d4f8.slice/crio-80684abcf6b082d636501c8b647983a0b4a17fe5fa32407fdba73d4d0a58a0f7 WatchSource:0}: Error finding container 80684abcf6b082d636501c8b647983a0b4a17fe5fa32407fdba73d4d0a58a0f7: Status 404 returned error can't find the container with id 80684abcf6b082d636501c8b647983a0b4a17fe5fa32407fdba73d4d0a58a0f7
Jul 22 12:43:54.768827 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:54.768809    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:43:55.393604 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:55.393569    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5" event={"ID":"41cc80f4-6019-4efd-8352-7b29cfe4d4f8","Type":"ContainerStarted","Data":"80684abcf6b082d636501c8b647983a0b4a17fe5fa32407fdba73d4d0a58a0f7"}
Jul 22 12:43:57.401566 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:57.401537    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5" event={"ID":"41cc80f4-6019-4efd-8352-7b29cfe4d4f8","Type":"ContainerStarted","Data":"effe8dc601590c24e09ffcab1b21ecbe888edc00eacf78d2da58e0f0918167d8"}
Jul 22 12:43:57.418669 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:43:57.418616    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-587h5" podStartSLOduration=0.902495899 podStartE2EDuration="3.418601305s" totalImagesPullingTime="2.516105406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:43:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:43:57.417674342 +0000 UTC m=+421.882572430" watchObservedRunningTime="2026-07-22 12:43:57.418601305 +0000 UTC m=+421.883499353"
Jul 22 12:44:18.166710 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.166635    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"]
Jul 22 12:44:18.169668 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.169654    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.174582 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.174559    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 22 12:44:18.174836 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.174817    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 22 12:44:18.174948 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.174883    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-22wzv\""
Jul 22 12:44:18.174948 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.174912    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 22 12:44:18.175059 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.174980    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 22 12:44:18.186997 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.186975    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"]
Jul 22 12:44:18.296542 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.296513    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.296666 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.296548    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mrbr\" (UniqueName: \"kubernetes.io/projected/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-kube-api-access-5mrbr\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.296666 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.296574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-tmp\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.296666 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.296617    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-manifests\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.296666 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.296644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.397061 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.397029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-manifests\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.397162 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.397072    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.397162 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.397097    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.397162 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.397117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mrbr\" (UniqueName: \"kubernetes.io/projected/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-kube-api-access-5mrbr\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.397162 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.397140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-tmp\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.397478 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.397453    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-manifests\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.399482 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.399463    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-tmp\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.399693 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.399676    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-webhook-cert\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.399842 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.399824    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.407206 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.407181    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mrbr\" (UniqueName: \"kubernetes.io/projected/f2c6eb62-befc-4117-a0ed-5cd14043d9f0-kube-api-access-5mrbr\") pod \"opendatahub-operator-controller-manager-d99b87cc7-s9h9f\" (UID: \"f2c6eb62-befc-4117-a0ed-5cd14043d9f0\") " pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.478944 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.478920    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:18.614875 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:18.614853    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"]
Jul 22 12:44:18.617905 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:18.617871    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2c6eb62_befc_4117_a0ed_5cd14043d9f0.slice/crio-bb8a405428962674cbee797a0d8b0f034c793479f566cd08cd54522b949cd0c5 WatchSource:0}: Error finding container bb8a405428962674cbee797a0d8b0f034c793479f566cd08cd54522b949cd0c5: Status 404 returned error can't find the container with id bb8a405428962674cbee797a0d8b0f034c793479f566cd08cd54522b949cd0c5
Jul 22 12:44:19.477831 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:19.477794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f" event={"ID":"f2c6eb62-befc-4117-a0ed-5cd14043d9f0","Type":"ContainerStarted","Data":"bb8a405428962674cbee797a0d8b0f034c793479f566cd08cd54522b949cd0c5"}
Jul 22 12:44:22.490822 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:22.490788    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f" event={"ID":"f2c6eb62-befc-4117-a0ed-5cd14043d9f0","Type":"ContainerStarted","Data":"9fbd805c6059b97234d09faf4d31acdc4a26467eec3e4eea144e9e8397d2ed28"}
Jul 22 12:44:23.495248 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:23.495209    2563 generic.go:356] "Generic (PLEG): container finished" podID="f2c6eb62-befc-4117-a0ed-5cd14043d9f0" containerID="9fbd805c6059b97234d09faf4d31acdc4a26467eec3e4eea144e9e8397d2ed28" exitCode=0
Jul 22 12:44:23.495684 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:23.495298    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f" event={"ID":"f2c6eb62-befc-4117-a0ed-5cd14043d9f0","Type":"ContainerDied","Data":"9fbd805c6059b97234d09faf4d31acdc4a26467eec3e4eea144e9e8397d2ed28"}
Jul 22 12:44:24.500186 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:24.500150    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f" event={"ID":"f2c6eb62-befc-4117-a0ed-5cd14043d9f0","Type":"ContainerStarted","Data":"fdc1df8abccdd16d3c1b4fbd5d6ecce31adf6a4637a1cb0000e2d4a412be4308"}
Jul 22 12:44:24.500588 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:24.500251    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:24.522147 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:24.522105    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f" podStartSLOduration=2.245266136 podStartE2EDuration="6.522091115s" totalImagesPullingTime="3.211816913s" totalInitContainerRuntime="1.065008066s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:24.521060162 +0000 UTC m=+448.985958210" watchObservedRunningTime="2026-07-22 12:44:24.522091115 +0000 UTC m=+448.986989163"
Jul 22 12:44:35.506046 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:35.506015    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d99b87cc7-s9h9f"
Jul 22 12:44:37.913058 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.913024    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77d54956c5-6296t"]
Jul 22 12:44:37.921681 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.921660    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:37.925400 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.925373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 22 12:44:37.925513 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.925419    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 22 12:44:37.926462 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.926443    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4qpqz\""
Jul 22 12:44:37.926544 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.926479    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 22 12:44:37.926786 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.926770    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 22 12:44:37.937326 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:37.937305    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d54956c5-6296t"]
Jul 22 12:44:38.021654 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.021630    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tls-certs\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.021826 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.021659    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wh4zw\" (UniqueName: \"kubernetes.io/projected/74ed1838-9d65-46a7-bfd3-cfafd39861cd-kube-api-access-wh4zw\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.021826 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.021678    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tmp\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.122546 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.122517    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tls-certs\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.122670 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.122558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wh4zw\" (UniqueName: \"kubernetes.io/projected/74ed1838-9d65-46a7-bfd3-cfafd39861cd-kube-api-access-wh4zw\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.122670 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.122578    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tmp\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.122670 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:38.122662    2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 22 12:44:38.122800 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:38.122727    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tls-certs podName:74ed1838-9d65-46a7-bfd3-cfafd39861cd nodeName:}" failed. No retries permitted until 2026-07-22 12:44:38.622707953 +0000 UTC m=+463.087605980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tls-certs") pod "kube-auth-proxy-77d54956c5-6296t" (UID: "74ed1838-9d65-46a7-bfd3-cfafd39861cd") : secret "kube-auth-proxy-tls" not found
Jul 22 12:44:38.124984 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.124960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tmp\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.138692 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.138673    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wh4zw\" (UniqueName: \"kubernetes.io/projected/74ed1838-9d65-46a7-bfd3-cfafd39861cd-kube-api-access-wh4zw\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.309936 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.309902    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ct2g7"]
Jul 22 12:44:38.313132 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.313117    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:38.316431 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.316410    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 22 12:44:38.316532 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.316518    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ww75z\""
Jul 22 12:44:38.324609 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.324589    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv655\" (UniqueName: \"kubernetes.io/projected/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-kube-api-access-mv655\") pod \"odh-model-controller-7998c67dc4-ct2g7\" (UID: \"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64\") " pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:38.324686 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.324622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-cert\") pod \"odh-model-controller-7998c67dc4-ct2g7\" (UID: \"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64\") " pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:38.345257 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.345234    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ct2g7"]
Jul 22 12:44:38.425814 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.425786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-cert\") pod \"odh-model-controller-7998c67dc4-ct2g7\" (UID: \"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64\") " pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:38.425951 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.425883    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv655\" (UniqueName: \"kubernetes.io/projected/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-kube-api-access-mv655\") pod \"odh-model-controller-7998c67dc4-ct2g7\" (UID: \"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64\") " pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:38.425951 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:38.425931    2563 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 22 12:44:38.426020 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:38.426001    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-cert podName:8d2eb7f9-faea-44dd-8f39-693e5bb0fe64 nodeName:}" failed. No retries permitted until 2026-07-22 12:44:38.925985589 +0000 UTC m=+463.390883619 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-cert") pod "odh-model-controller-7998c67dc4-ct2g7" (UID: "8d2eb7f9-faea-44dd-8f39-693e5bb0fe64") : secret "odh-model-controller-webhook-cert" not found
Jul 22 12:44:38.436490 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.436460    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv655\" (UniqueName: \"kubernetes.io/projected/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-kube-api-access-mv655\") pod \"odh-model-controller-7998c67dc4-ct2g7\" (UID: \"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64\") " pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:38.627193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.627101    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tls-certs\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.629545 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.629528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/74ed1838-9d65-46a7-bfd3-cfafd39861cd-tls-certs\") pod \"kube-auth-proxy-77d54956c5-6296t\" (UID: \"74ed1838-9d65-46a7-bfd3-cfafd39861cd\") " pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.830790 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.830734    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t"
Jul 22 12:44:38.929340 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.929311    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-cert\") pod \"odh-model-controller-7998c67dc4-ct2g7\" (UID: \"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64\") " pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:38.931646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:38.931624    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8d2eb7f9-faea-44dd-8f39-693e5bb0fe64-cert\") pod \"odh-model-controller-7998c67dc4-ct2g7\" (UID: \"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64\") " pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:39.002893 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.002865    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77d54956c5-6296t"]
Jul 22 12:44:39.003666 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:39.003640    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74ed1838_9d65_46a7_bfd3_cfafd39861cd.slice/crio-60899afac31689e97f1d7e67461aabcfe4ec772183e843fd995babcb320644bd WatchSource:0}: Error finding container 60899afac31689e97f1d7e67461aabcfe4ec772183e843fd995babcb320644bd: Status 404 returned error can't find the container with id 60899afac31689e97f1d7e67461aabcfe4ec772183e843fd995babcb320644bd
Jul 22 12:44:39.223212 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.223186    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:39.223353 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.223286    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-krbjc"]
Jul 22 12:44:39.227570 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.227555    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:39.231072 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.231051    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 22 12:44:39.232472 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.232443    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqwb9\" (UniqueName: \"kubernetes.io/projected/a77a4148-91cc-4838-bf15-d2dfa9357c7d-kube-api-access-kqwb9\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:39.232567 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.232500    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:39.249011 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.248988    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-krbjc"]
Jul 22 12:44:39.333354 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.333328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqwb9\" (UniqueName: \"kubernetes.io/projected/a77a4148-91cc-4838-bf15-d2dfa9357c7d-kube-api-access-kqwb9\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:39.333469 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.333401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:39.333516 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:39.333495    2563 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 22 12:44:39.333560 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:39.333550    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs podName:a77a4148-91cc-4838-bf15-d2dfa9357c7d nodeName:}" failed. No retries permitted until 2026-07-22 12:44:39.833534852 +0000 UTC m=+464.298432892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs") pod "model-serving-api-54d89bd5fc-krbjc" (UID: "a77a4148-91cc-4838-bf15-d2dfa9357c7d") : secret "model-serving-api-tls" not found
Jul 22 12:44:39.342476 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.342450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqwb9\" (UniqueName: \"kubernetes.io/projected/a77a4148-91cc-4838-bf15-d2dfa9357c7d-kube-api-access-kqwb9\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:39.351864 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.351841    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-ct2g7"]
Jul 22 12:44:39.354739 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:39.354715    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2eb7f9_faea_44dd_8f39_693e5bb0fe64.slice/crio-12fdfbf8e17875045676fa3aa14026ecab5ff970c45f5b2fefef3f24e1e12cdc WatchSource:0}: Error finding container 12fdfbf8e17875045676fa3aa14026ecab5ff970c45f5b2fefef3f24e1e12cdc: Status 404 returned error can't find the container with id 12fdfbf8e17875045676fa3aa14026ecab5ff970c45f5b2fefef3f24e1e12cdc
Jul 22 12:44:39.509309 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.509238    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"]
Jul 22 12:44:39.513626 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.513610    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.516419 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.516397    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 22 12:44:39.516540 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.516461    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6pb2j\""
Jul 22 12:44:39.516677 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.516664    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 22 12:44:39.516740 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.516664    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 22 12:44:39.516958 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.516943    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 22 12:44:39.526827 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.526804    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"]
Jul 22 12:44:39.536016 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.535986    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.536115 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.536025    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.536152 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.536126    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2554b5-138b-4b6e-a7d8-e60761e64a78-local-certs\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.536193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.536176    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.536227 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.536208    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-cacerts\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.536274 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.536262    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-token\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.536311 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.536284    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x22xz\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-kube-api-access-x22xz\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.553210 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.553182    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7" event={"ID":"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64","Type":"ContainerStarted","Data":"12fdfbf8e17875045676fa3aa14026ecab5ff970c45f5b2fefef3f24e1e12cdc"}
Jul 22 12:44:39.554241 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.554222    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t" event={"ID":"74ed1838-9d65-46a7-bfd3-cfafd39861cd","Type":"ContainerStarted","Data":"60899afac31689e97f1d7e67461aabcfe4ec772183e843fd995babcb320644bd"}
Jul 22 12:44:39.636918 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.636881    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.637066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.636981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2554b5-138b-4b6e-a7d8-e60761e64a78-local-certs\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.637066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.637016    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.637066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.637047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-cacerts\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.637218 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.637095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-token\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.637218 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.637122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x22xz\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-kube-api-access-x22xz\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.637218 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.637154    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.637766 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.637721    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.640049 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.640028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2554b5-138b-4b6e-a7d8-e60761e64a78-local-certs\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.640189 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.640117    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.640257 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.640129    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.640297 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.640275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-cacerts\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.647386 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.647367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-token\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.647477 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.647432    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x22xz\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-kube-api-access-x22xz\") pod \"istiod-openshift-gateway-7574459656-9tzpc\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.822840 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.822741    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:39.838532 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.838508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:39.838667 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:39.838652    2563 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found
Jul 22 12:44:39.838768 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:39.838733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs podName:a77a4148-91cc-4838-bf15-d2dfa9357c7d nodeName:}" failed. No retries permitted until 2026-07-22 12:44:40.838711479 +0000 UTC m=+465.303609513 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs") pod "model-serving-api-54d89bd5fc-krbjc" (UID: "a77a4148-91cc-4838-bf15-d2dfa9357c7d") : secret "model-serving-api-tls" not found
Jul 22 12:44:39.979261 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:39.979079    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"]
Jul 22 12:44:39.982486 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:39.982448    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e2554b5_138b_4b6e_a7d8_e60761e64a78.slice/crio-87ec0b4137c8fb67c362f7b05320247fbc564035a9973bc042f1874afd627cdf WatchSource:0}: Error finding container 87ec0b4137c8fb67c362f7b05320247fbc564035a9973bc042f1874afd627cdf: Status 404 returned error can't find the container with id 87ec0b4137c8fb67c362f7b05320247fbc564035a9973bc042f1874afd627cdf
Jul 22 12:44:40.560857 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:40.560816    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" event={"ID":"3e2554b5-138b-4b6e-a7d8-e60761e64a78","Type":"ContainerStarted","Data":"87ec0b4137c8fb67c362f7b05320247fbc564035a9973bc042f1874afd627cdf"}
Jul 22 12:44:40.848060 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:40.847954    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:40.851880 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:40.851853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a77a4148-91cc-4838-bf15-d2dfa9357c7d-tls-certs\") pod \"model-serving-api-54d89bd5fc-krbjc\" (UID: \"a77a4148-91cc-4838-bf15-d2dfa9357c7d\") " pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:41.050055 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:41.049707    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:43.865193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:43.865143    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:44:43.865562 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:43.865246    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:44:43.942968 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:43.942946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-krbjc"]
Jul 22 12:44:43.945661 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:43.945639    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda77a4148_91cc_4838_bf15_d2dfa9357c7d.slice/crio-1012941d81cb4dcb6d8de111a8eb422a256459aa96583401e44847a667abb0d1 WatchSource:0}: Error finding container 1012941d81cb4dcb6d8de111a8eb422a256459aa96583401e44847a667abb0d1: Status 404 returned error can't find the container with id 1012941d81cb4dcb6d8de111a8eb422a256459aa96583401e44847a667abb0d1
Jul 22 12:44:44.580640 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.580594    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7" event={"ID":"8d2eb7f9-faea-44dd-8f39-693e5bb0fe64","Type":"ContainerStarted","Data":"c6aed846c18bdb6169abae7e7d6156e1027c03ffb38974f3b7bd3bcae06dfdd0"}
Jul 22 12:44:44.580869 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.580743    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:44.582304 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.582276    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t" event={"ID":"74ed1838-9d65-46a7-bfd3-cfafd39861cd","Type":"ContainerStarted","Data":"f691e05f0ada4e7e5305215c91e9fc29f6907e7ac0566ff972030d8bdde8fa1a"}
Jul 22 12:44:44.583517 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.583499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-krbjc" event={"ID":"a77a4148-91cc-4838-bf15-d2dfa9357c7d","Type":"ContainerStarted","Data":"1012941d81cb4dcb6d8de111a8eb422a256459aa96583401e44847a667abb0d1"}
Jul 22 12:44:44.585034 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.585012    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" event={"ID":"3e2554b5-138b-4b6e-a7d8-e60761e64a78","Type":"ContainerStarted","Data":"5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a"}
Jul 22 12:44:44.585226 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.585210    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:44.586958 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.586934    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-9tzpc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:44:44.587049 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.586997    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" podUID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:44:44.604855 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.604814    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7" podStartSLOduration=2.152435339 podStartE2EDuration="6.604800333s" totalImagesPullingTime="4.452364994s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:44.603766125 +0000 UTC m=+469.068664170" watchObservedRunningTime="2026-07-22 12:44:44.604800333 +0000 UTC m=+469.069698381"
Jul 22 12:44:44.625028 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.624977    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" podStartSLOduration=1.745021485 podStartE2EDuration="5.624962007s" totalImagesPullingTime="3.879940522s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:44.623391858 +0000 UTC m=+469.088289907" watchObservedRunningTime="2026-07-22 12:44:44.624962007 +0000 UTC m=+469.089860055"
Jul 22 12:44:44.645501 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:44.645445    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77d54956c5-6296t" podStartSLOduration=2.850030724 podStartE2EDuration="7.645426227s" totalImagesPullingTime="4.795395503s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:44.643615401 +0000 UTC m=+469.108513461" watchObservedRunningTime="2026-07-22 12:44:44.645426227 +0000 UTC m=+469.110324278"
Jul 22 12:44:45.590647 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.590613    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-9tzpc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 12:44:45.591190 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.590704    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" podUID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 12:44:45.797377 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.797347    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"]
Jul 22 12:44:45.800533 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.800515    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.805370 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.805345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-gzd75\""
Jul 22 12:44:45.817965 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.817938    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"]
Jul 22 12:44:45.888860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.888778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.888860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.888831    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.888860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.888856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.889066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.888875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kccwm\" (UniqueName: \"kubernetes.io/projected/b89650cf-0d00-4bd4-b26f-5d1316817e5c-kube-api-access-kccwm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.889066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.888900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.889066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.888948    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.889066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.888982    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.889066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.889012    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.889066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.889037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.989927 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.989884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.989927 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.989921    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990171 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.989941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990171 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.989974    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990171 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990013    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990171 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990060    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990171 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990103    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990171 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990171 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kccwm\" (UniqueName: \"kubernetes.io/projected/b89650cf-0d00-4bd4-b26f-5d1316817e5c-kube-api-access-kccwm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990488 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990348    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990488 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990373    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990488 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990451    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990730 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.990809 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.990739    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.992653 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.992633    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:45.992653 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:45.992649    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:46.010312 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.010288    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b89650cf-0d00-4bd4-b26f-5d1316817e5c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:46.010535 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.010518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kccwm\" (UniqueName: \"kubernetes.io/projected/b89650cf-0d00-4bd4-b26f-5d1316817e5c-kube-api-access-kccwm\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4xx2dp\" (UID: \"b89650cf-0d00-4bd4-b26f-5d1316817e5c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:46.110556 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.110523    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:46.258585 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.258557    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"]
Jul 22 12:44:46.261093 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:46.261067    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb89650cf_0d00_4bd4_b26f_5d1316817e5c.slice/crio-8c821e7baf349ef77d241d1ab0e32c3c1bf909c73a01c075148afc5394504893 WatchSource:0}: Error finding container 8c821e7baf349ef77d241d1ab0e32c3c1bf909c73a01c075148afc5394504893: Status 404 returned error can't find the container with id 8c821e7baf349ef77d241d1ab0e32c3c1bf909c73a01c075148afc5394504893
Jul 22 12:44:46.596483 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.596443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp" event={"ID":"b89650cf-0d00-4bd4-b26f-5d1316817e5c","Type":"ContainerStarted","Data":"8c821e7baf349ef77d241d1ab0e32c3c1bf909c73a01c075148afc5394504893"}
Jul 22 12:44:46.597883 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.597852    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-krbjc" event={"ID":"a77a4148-91cc-4838-bf15-d2dfa9357c7d","Type":"ContainerStarted","Data":"52e38c9478bd42fe20e157d8c81d9f47f5b241ae36fded6bfdb760920e090cf9"}
Jul 22 12:44:46.598021 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.598007    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:44:46.620911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:46.620863    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-krbjc" podStartSLOduration=5.866554884 podStartE2EDuration="7.620846128s" totalImagesPullingTime="1.754291244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:46.619091199 +0000 UTC m=+471.083989249" watchObservedRunningTime="2026-07-22 12:44:46.620846128 +0000 UTC m=+471.085744176"
Jul 22 12:44:48.590812 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.590779    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:48.599356 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.599329    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"]
Jul 22 12:44:48.606144 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.606120    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.609364 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.609272    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 22 12:44:48.609965 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.609942    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-j2lxr\""
Jul 22 12:44:48.610830 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.610805    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"]
Jul 22 12:44:48.615668 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.615644    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:48.615733 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.615719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"]
Jul 22 12:44:48.619421 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.619153    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 22 12:44:48.619421 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.619217    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ctb2l\""
Jul 22 12:44:48.641938 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.641901    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"]
Jul 22 12:44:48.714942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.714902    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbkkr\" (UniqueName: \"kubernetes.io/projected/f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9-kube-api-access-kbkkr\") pod \"llmisvc-controller-manager-777c6bd7d5-s2kl7\" (UID: \"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:48.715124 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.714960    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-s2kl7\" (UID: \"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:48.715124 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.714992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/400cffb6-63ed-4f65-96b8-ad1dfbdfea02-cert\") pod \"kserve-controller-manager-d6f4b65b8-k2qvn\" (UID: \"400cffb6-63ed-4f65-96b8-ad1dfbdfea02\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.715124 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.715094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4ddt\" (UniqueName: \"kubernetes.io/projected/400cffb6-63ed-4f65-96b8-ad1dfbdfea02-kube-api-access-l4ddt\") pod \"kserve-controller-manager-d6f4b65b8-k2qvn\" (UID: \"400cffb6-63ed-4f65-96b8-ad1dfbdfea02\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.816216 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.816181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbkkr\" (UniqueName: \"kubernetes.io/projected/f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9-kube-api-access-kbkkr\") pod \"llmisvc-controller-manager-777c6bd7d5-s2kl7\" (UID: \"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:48.816216 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.816221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-s2kl7\" (UID: \"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:48.816485 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.816256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/400cffb6-63ed-4f65-96b8-ad1dfbdfea02-cert\") pod \"kserve-controller-manager-d6f4b65b8-k2qvn\" (UID: \"400cffb6-63ed-4f65-96b8-ad1dfbdfea02\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.816485 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.816307    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4ddt\" (UniqueName: \"kubernetes.io/projected/400cffb6-63ed-4f65-96b8-ad1dfbdfea02-kube-api-access-l4ddt\") pod \"kserve-controller-manager-d6f4b65b8-k2qvn\" (UID: \"400cffb6-63ed-4f65-96b8-ad1dfbdfea02\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.818862 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.818832    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/400cffb6-63ed-4f65-96b8-ad1dfbdfea02-cert\") pod \"kserve-controller-manager-d6f4b65b8-k2qvn\" (UID: \"400cffb6-63ed-4f65-96b8-ad1dfbdfea02\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.818973 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.818924    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-s2kl7\" (UID: \"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:48.828809 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.828780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbkkr\" (UniqueName: \"kubernetes.io/projected/f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9-kube-api-access-kbkkr\") pod \"llmisvc-controller-manager-777c6bd7d5-s2kl7\" (UID: \"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:48.831300 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.831277    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4ddt\" (UniqueName: \"kubernetes.io/projected/400cffb6-63ed-4f65-96b8-ad1dfbdfea02-kube-api-access-l4ddt\") pod \"kserve-controller-manager-d6f4b65b8-k2qvn\" (UID: \"400cffb6-63ed-4f65-96b8-ad1dfbdfea02\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.895844 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.895815    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:44:48.895918 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.895883    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:44:48.895918 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.895910    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:44:48.920507 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.920475    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:48.930596 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:48.930571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:49.096723 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.096699    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"]
Jul 22 12:44:49.099858 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:49.099828    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400cffb6_63ed_4f65_96b8_ad1dfbdfea02.slice/crio-d3e2f6f3a6e3ee02cedeb11625597bdbab6606c719de629847b9b2d6c62aced0 WatchSource:0}: Error finding container d3e2f6f3a6e3ee02cedeb11625597bdbab6606c719de629847b9b2d6c62aced0: Status 404 returned error can't find the container with id d3e2f6f3a6e3ee02cedeb11625597bdbab6606c719de629847b9b2d6c62aced0
Jul 22 12:44:49.131933 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.131856    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"]
Jul 22 12:44:49.135790 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:44:49.135735    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf5f2df0b_dc00_4193_8dc1_8eda4bec4ea9.slice/crio-861182444d86bb8e7941d4b5d3694f952bd12ba66925a27802ec478a96e1c3d0 WatchSource:0}: Error finding container 861182444d86bb8e7941d4b5d3694f952bd12ba66925a27802ec478a96e1c3d0: Status 404 returned error can't find the container with id 861182444d86bb8e7941d4b5d3694f952bd12ba66925a27802ec478a96e1c3d0
Jul 22 12:44:49.613055 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.613022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7" event={"ID":"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9","Type":"ContainerStarted","Data":"861182444d86bb8e7941d4b5d3694f952bd12ba66925a27802ec478a96e1c3d0"}
Jul 22 12:44:49.614227 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.614196    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn" event={"ID":"400cffb6-63ed-4f65-96b8-ad1dfbdfea02","Type":"ContainerStarted","Data":"d3e2f6f3a6e3ee02cedeb11625597bdbab6606c719de629847b9b2d6c62aced0"}
Jul 22 12:44:49.615737 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.615718    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp" event={"ID":"b89650cf-0d00-4bd4-b26f-5d1316817e5c","Type":"ContainerStarted","Data":"c617e4208a75576f8d6c28d570d7229d0a28e9f460dba4f3d04a608a515f8f47"}
Jul 22 12:44:49.640552 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.640494    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp" podStartSLOduration=2.007972972 podStartE2EDuration="4.640475606s" totalImagesPullingTime="2.632502634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:49.639998631 +0000 UTC m=+474.104896680" watchObservedRunningTime="2026-07-22 12:44:49.640475606 +0000 UTC m=+474.105373636"
Jul 22 12:44:49.921437 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.921346    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"]
Jul 22 12:44:49.921646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:49.921616    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" podUID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" containerName="discovery" containerID="cri-o://5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a" gracePeriod=30
Jul 22 12:44:50.110940 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.110913    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:50.118589 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.117811    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:50.260341 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.260312    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:50.332105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.331556    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-kubeconfig\") pod \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") "
Jul 22 12:44:50.332105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.331607    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2554b5-138b-4b6e-a7d8-e60761e64a78-local-certs\") pod \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") "
Jul 22 12:44:50.332105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.331643    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-dns-cert\") pod \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") "
Jul 22 12:44:50.332105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.331685    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-cacerts\") pod \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") "
Jul 22 12:44:50.332105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.331727    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-token\") pod \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") "
Jul 22 12:44:50.332105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.331818    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-ca-configmap\") pod \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") "
Jul 22 12:44:50.332105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.331843    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x22xz\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-kube-api-access-x22xz\") pod \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\" (UID: \"3e2554b5-138b-4b6e-a7d8-e60761e64a78\") "
Jul 22 12:44:50.332586 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.332406    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "3e2554b5-138b-4b6e-a7d8-e60761e64a78" (UID: "3e2554b5-138b-4b6e-a7d8-e60761e64a78"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:44:50.335055 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.335016    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-token" (OuterVolumeSpecName: "istio-token") pod "3e2554b5-138b-4b6e-a7d8-e60761e64a78" (UID: "3e2554b5-138b-4b6e-a7d8-e60761e64a78"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:44:50.335300 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.335195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "3e2554b5-138b-4b6e-a7d8-e60761e64a78" (UID: "3e2554b5-138b-4b6e-a7d8-e60761e64a78"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:44:50.336115 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.336095    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e2554b5-138b-4b6e-a7d8-e60761e64a78-local-certs" (OuterVolumeSpecName: "local-certs") pod "3e2554b5-138b-4b6e-a7d8-e60761e64a78" (UID: "3e2554b5-138b-4b6e-a7d8-e60761e64a78"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:44:50.336213 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.336157    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "3e2554b5-138b-4b6e-a7d8-e60761e64a78" (UID: "3e2554b5-138b-4b6e-a7d8-e60761e64a78"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:44:50.336213 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.336158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-kube-api-access-x22xz" (OuterVolumeSpecName: "kube-api-access-x22xz") pod "3e2554b5-138b-4b6e-a7d8-e60761e64a78" (UID: "3e2554b5-138b-4b6e-a7d8-e60761e64a78"). InnerVolumeSpecName "kube-api-access-x22xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:44:50.336443 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.336417    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-cacerts" (OuterVolumeSpecName: "cacerts") pod "3e2554b5-138b-4b6e-a7d8-e60761e64a78" (UID: "3e2554b5-138b-4b6e-a7d8-e60761e64a78"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:44:50.433203 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.433169    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-kubeconfig\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:44:50.433203 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.433205    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e2554b5-138b-4b6e-a7d8-e60761e64a78-local-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:44:50.433413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.433224    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-dns-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:44:50.433413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.433243    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/3e2554b5-138b-4b6e-a7d8-e60761e64a78-cacerts\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:44:50.433413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.433258    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:44:50.433413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.433276    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/3e2554b5-138b-4b6e-a7d8-e60761e64a78-istio-csr-ca-configmap\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:44:50.433413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.433289    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x22xz\" (UniqueName: \"kubernetes.io/projected/3e2554b5-138b-4b6e-a7d8-e60761e64a78-kube-api-access-x22xz\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:44:50.631962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.630866    2563 generic.go:356] "Generic (PLEG): container finished" podID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" containerID="5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a" exitCode=0
Jul 22 12:44:50.631962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.630945    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"
Jul 22 12:44:50.631962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.631015    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" event={"ID":"3e2554b5-138b-4b6e-a7d8-e60761e64a78","Type":"ContainerDied","Data":"5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a"}
Jul 22 12:44:50.631962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.631086    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc" event={"ID":"3e2554b5-138b-4b6e-a7d8-e60761e64a78","Type":"ContainerDied","Data":"87ec0b4137c8fb67c362f7b05320247fbc564035a9973bc042f1874afd627cdf"}
Jul 22 12:44:50.631962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.631111    2563 scope.go:117] "RemoveContainer" containerID="5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a"
Jul 22 12:44:50.631962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.631921    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:50.637796 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.637769    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4xx2dp"
Jul 22 12:44:50.643991 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.643974    2563 scope.go:117] "RemoveContainer" containerID="5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a"
Jul 22 12:44:50.644324 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:44:50.644297    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a\": container with ID starting with 5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a not found: ID does not exist" containerID="5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a"
Jul 22 12:44:50.644434 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.644336    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a"} err="failed to get container status \"5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a\": rpc error: code = NotFound desc = could not find container \"5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a\": container with ID starting with 5286250a1ee06863879b8d303445a538aaa7843f31473743f0e08f92d504de8a not found: ID does not exist"
Jul 22 12:44:50.676311 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.676246    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"]
Jul 22 12:44:50.689813 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:50.689727    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-9tzpc"]
Jul 22 12:44:51.639313 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:51.638912    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn" event={"ID":"400cffb6-63ed-4f65-96b8-ad1dfbdfea02","Type":"ContainerStarted","Data":"8da42c4780d170f08ec997e3d0ac4c6bfe76ec15659a854ee7ef9234298ebce4"}
Jul 22 12:44:51.639313 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:51.639271    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:44:51.658699 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:51.658644    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn" podStartSLOduration=1.447453423 podStartE2EDuration="3.658622699s" totalImagesPullingTime="2.211169276s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:51.657874417 +0000 UTC m=+476.122772492" watchObservedRunningTime="2026-07-22 12:44:51.658622699 +0000 UTC m=+476.123520749"
Jul 22 12:44:52.019098 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:52.019065    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" path="/var/lib/kubelet/pods/3e2554b5-138b-4b6e-a7d8-e60761e64a78/volumes"
Jul 22 12:44:52.643776 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:52.643725    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7" event={"ID":"f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9","Type":"ContainerStarted","Data":"c3889011228c867732b9cc016c33ca1ed5dd8ffacf497ada6a0ad388b8cba548"}
Jul 22 12:44:52.644233 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:52.643988    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:44:52.663344 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:52.663296    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7" podStartSLOduration=1.6157724679999999 podStartE2EDuration="4.663282676s" totalImagesPullingTime="3.047510208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:44:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:44:52.661995585 +0000 UTC m=+477.126893626" watchObservedRunningTime="2026-07-22 12:44:52.663282676 +0000 UTC m=+477.128180727"
Jul 22 12:44:55.592428 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:55.592401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-ct2g7"
Jul 22 12:44:57.607616 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:44:57.607585    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-krbjc"
Jul 22 12:45:22.648972 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:22.648943    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-k2qvn"
Jul 22 12:45:23.649873 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:23.649848    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-s2kl7"
Jul 22 12:45:39.090801 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.090772    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jxg5s"]
Jul 22 12:45:39.091147 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.091065    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" containerName="discovery"
Jul 22 12:45:39.091147 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.091075    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" containerName="discovery"
Jul 22 12:45:39.091147 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.091130    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3e2554b5-138b-4b6e-a7d8-e60761e64a78" containerName="discovery"
Jul 22 12:45:39.098387 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.098358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s"
Jul 22 12:45:39.101087 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.101055    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 22 12:45:39.101477 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.101457    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 22 12:45:39.102470 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.102445    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-4nt49\""
Jul 22 12:45:39.105209 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.105188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jxg5s"]
Jul 22 12:45:39.204626 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.204596    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lmwn\" (UniqueName: \"kubernetes.io/projected/3ec1149e-efc2-483d-94fa-e0360bd703a7-kube-api-access-4lmwn\") pod \"kuadrant-operator-catalog-jxg5s\" (UID: \"3ec1149e-efc2-483d-94fa-e0360bd703a7\") " pod="kuadrant-system/kuadrant-operator-catalog-jxg5s"
Jul 22 12:45:39.305812 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.305790    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lmwn\" (UniqueName: \"kubernetes.io/projected/3ec1149e-efc2-483d-94fa-e0360bd703a7-kube-api-access-4lmwn\") pod \"kuadrant-operator-catalog-jxg5s\" (UID: \"3ec1149e-efc2-483d-94fa-e0360bd703a7\") " pod="kuadrant-system/kuadrant-operator-catalog-jxg5s"
Jul 22 12:45:39.317183 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.317156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lmwn\" (UniqueName: \"kubernetes.io/projected/3ec1149e-efc2-483d-94fa-e0360bd703a7-kube-api-access-4lmwn\") pod \"kuadrant-operator-catalog-jxg5s\" (UID: \"3ec1149e-efc2-483d-94fa-e0360bd703a7\") " pod="kuadrant-system/kuadrant-operator-catalog-jxg5s"
Jul 22 12:45:39.411231 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.411174    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s"
Jul 22 12:45:39.454288 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.454260    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jxg5s"]
Jul 22 12:45:39.537190 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.537168    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jxg5s"]
Jul 22 12:45:39.539412 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:45:39.539384    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ec1149e_efc2_483d_94fa_e0360bd703a7.slice/crio-1ba1390c90a50241542535a36478988d5abb7632e14c469a51fe5c618ef49ca1 WatchSource:0}: Error finding container 1ba1390c90a50241542535a36478988d5abb7632e14c469a51fe5c618ef49ca1: Status 404 returned error can't find the container with id 1ba1390c90a50241542535a36478988d5abb7632e14c469a51fe5c618ef49ca1
Jul 22 12:45:39.670339 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.670261    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77w8d"]
Jul 22 12:45:39.672997 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.672978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:39.681598 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.681581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77w8d"]
Jul 22 12:45:39.811020 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.810987    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wj5c6\" (UniqueName: \"kubernetes.io/projected/28425d84-d69e-4b45-b430-541441021e0d-kube-api-access-wj5c6\") pod \"kuadrant-operator-catalog-77w8d\" (UID: \"28425d84-d69e-4b45-b430-541441021e0d\") " pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:39.834331 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.834303    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s" event={"ID":"3ec1149e-efc2-483d-94fa-e0360bd703a7","Type":"ContainerStarted","Data":"1ba1390c90a50241542535a36478988d5abb7632e14c469a51fe5c618ef49ca1"}
Jul 22 12:45:39.911572 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.911549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wj5c6\" (UniqueName: \"kubernetes.io/projected/28425d84-d69e-4b45-b430-541441021e0d-kube-api-access-wj5c6\") pod \"kuadrant-operator-catalog-77w8d\" (UID: \"28425d84-d69e-4b45-b430-541441021e0d\") " pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:39.919599 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.919578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wj5c6\" (UniqueName: \"kubernetes.io/projected/28425d84-d69e-4b45-b430-541441021e0d-kube-api-access-wj5c6\") pod \"kuadrant-operator-catalog-77w8d\" (UID: \"28425d84-d69e-4b45-b430-541441021e0d\") " pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:39.982689 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:39.982668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:40.107096 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:40.107072    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-77w8d"]
Jul 22 12:45:40.108895 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:45:40.108869    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28425d84_d69e_4b45_b430_541441021e0d.slice/crio-305a62160cddac3553da0f161bb3a232a80c970a68a4e861f1c081d3cd161ffa WatchSource:0}: Error finding container 305a62160cddac3553da0f161bb3a232a80c970a68a4e861f1c081d3cd161ffa: Status 404 returned error can't find the container with id 305a62160cddac3553da0f161bb3a232a80c970a68a4e861f1c081d3cd161ffa
Jul 22 12:45:40.840067 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:40.840037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-77w8d" event={"ID":"28425d84-d69e-4b45-b430-541441021e0d","Type":"ContainerStarted","Data":"305a62160cddac3553da0f161bb3a232a80c970a68a4e861f1c081d3cd161ffa"}
Jul 22 12:45:42.849971 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:42.849938    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s" event={"ID":"3ec1149e-efc2-483d-94fa-e0360bd703a7","Type":"ContainerStarted","Data":"7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8"}
Jul 22 12:45:42.850411 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:42.850021    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s" podUID="3ec1149e-efc2-483d-94fa-e0360bd703a7" containerName="registry-server" containerID="cri-o://7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8" gracePeriod=2
Jul 22 12:45:42.851955 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:42.851921    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-77w8d" event={"ID":"28425d84-d69e-4b45-b430-541441021e0d","Type":"ContainerStarted","Data":"eafc2c5c04faa554e7febf754ae159aed1546a5afadfe461334a4d66da080651"}
Jul 22 12:45:42.867380 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:42.867340    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s" podStartSLOduration=1.090841342 podStartE2EDuration="3.867326681s" totalImagesPullingTime="2.776485339s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:45:42.866134292 +0000 UTC m=+527.331032352" watchObservedRunningTime="2026-07-22 12:45:42.867326681 +0000 UTC m=+527.332224749"
Jul 22 12:45:42.898676 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:42.898639    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-77w8d" podStartSLOduration=1.694946257 podStartE2EDuration="3.898625074s" totalImagesPullingTime="2.203678817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:45:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:45:42.89503125 +0000 UTC m=+527.359929300" watchObservedRunningTime="2026-07-22 12:45:42.898625074 +0000 UTC m=+527.363523123"
Jul 22 12:45:43.085655 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.085631    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s"
Jul 22 12:45:43.241457 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.241435    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lmwn\" (UniqueName: \"kubernetes.io/projected/3ec1149e-efc2-483d-94fa-e0360bd703a7-kube-api-access-4lmwn\") pod \"3ec1149e-efc2-483d-94fa-e0360bd703a7\" (UID: \"3ec1149e-efc2-483d-94fa-e0360bd703a7\") "
Jul 22 12:45:43.243683 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.243653    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ec1149e-efc2-483d-94fa-e0360bd703a7-kube-api-access-4lmwn" (OuterVolumeSpecName: "kube-api-access-4lmwn") pod "3ec1149e-efc2-483d-94fa-e0360bd703a7" (UID: "3ec1149e-efc2-483d-94fa-e0360bd703a7"). InnerVolumeSpecName "kube-api-access-4lmwn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:45:43.342843 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.342824    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lmwn\" (UniqueName: \"kubernetes.io/projected/3ec1149e-efc2-483d-94fa-e0360bd703a7-kube-api-access-4lmwn\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:45:43.857249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.857216    2563 generic.go:356] "Generic (PLEG): container finished" podID="3ec1149e-efc2-483d-94fa-e0360bd703a7" containerID="7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8" exitCode=0
Jul 22 12:45:43.857663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.857262    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s"
Jul 22 12:45:43.857663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.857311    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s" event={"ID":"3ec1149e-efc2-483d-94fa-e0360bd703a7","Type":"ContainerDied","Data":"7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8"}
Jul 22 12:45:43.857663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.857371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jxg5s" event={"ID":"3ec1149e-efc2-483d-94fa-e0360bd703a7","Type":"ContainerDied","Data":"1ba1390c90a50241542535a36478988d5abb7632e14c469a51fe5c618ef49ca1"}
Jul 22 12:45:43.857663 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.857401    2563 scope.go:117] "RemoveContainer" containerID="7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8"
Jul 22 12:45:43.866536 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.866516    2563 scope.go:117] "RemoveContainer" containerID="7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8"
Jul 22 12:45:43.866790 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:45:43.866772    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8\": container with ID starting with 7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8 not found: ID does not exist" containerID="7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8"
Jul 22 12:45:43.866842 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.866798    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8"} err="failed to get container status \"7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8\": rpc error: code = NotFound desc = could not find container \"7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8\": container with ID starting with 7ec9cb47ff9793537723b5589a2a358f5a03f3098e5e9d16ea0cbb507bab2ac8 not found: ID does not exist"
Jul 22 12:45:43.878541 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.878511    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jxg5s"]
Jul 22 12:45:43.882080 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:43.882064    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jxg5s"]
Jul 22 12:45:44.016795 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:44.016742    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ec1149e-efc2-483d-94fa-e0360bd703a7" path="/var/lib/kubelet/pods/3ec1149e-efc2-483d-94fa-e0360bd703a7/volumes"
Jul 22 12:45:49.983736 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:49.983706    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:49.984221 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:49.983875    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:50.006165 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:50.006144    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:45:50.906181 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:45:50.906153    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-77w8d"
Jul 22 12:46:11.562969 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.562936    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"]
Jul 22 12:46:11.563349 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.563216    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ec1149e-efc2-483d-94fa-e0360bd703a7" containerName="registry-server"
Jul 22 12:46:11.563349 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.563226    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ec1149e-efc2-483d-94fa-e0360bd703a7" containerName="registry-server"
Jul 22 12:46:11.563349 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.563276    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ec1149e-efc2-483d-94fa-e0360bd703a7" containerName="registry-server"
Jul 22 12:46:11.567349 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.567333    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"
Jul 22 12:46:11.586264 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.586239    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lkpz8\""
Jul 22 12:46:11.624689 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.624663    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"]
Jul 22 12:46:11.641243 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.641219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nkwv\" (UniqueName: \"kubernetes.io/projected/3277f9b3-452f-4185-847f-5a8fb6e1aec5-kube-api-access-4nkwv\") pod \"authorino-operator-75cc8d5cb7-mqp5k\" (UID: \"3277f9b3-452f-4185-847f-5a8fb6e1aec5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"
Jul 22 12:46:11.742488 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.742463    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nkwv\" (UniqueName: \"kubernetes.io/projected/3277f9b3-452f-4185-847f-5a8fb6e1aec5-kube-api-access-4nkwv\") pod \"authorino-operator-75cc8d5cb7-mqp5k\" (UID: \"3277f9b3-452f-4185-847f-5a8fb6e1aec5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"
Jul 22 12:46:11.759550 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.759527    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nkwv\" (UniqueName: \"kubernetes.io/projected/3277f9b3-452f-4185-847f-5a8fb6e1aec5-kube-api-access-4nkwv\") pod \"authorino-operator-75cc8d5cb7-mqp5k\" (UID: \"3277f9b3-452f-4185-847f-5a8fb6e1aec5\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"
Jul 22 12:46:11.876374 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:11.876320    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"
Jul 22 12:46:12.036588 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:12.036540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"]
Jul 22 12:46:12.038777 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:46:12.038738    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3277f9b3_452f_4185_847f_5a8fb6e1aec5.slice/crio-ec591b3bdb6a69f8c2f202584920b6722fd9896ffe58f4781d17cefe35e8543c WatchSource:0}: Error finding container ec591b3bdb6a69f8c2f202584920b6722fd9896ffe58f4781d17cefe35e8543c: Status 404 returned error can't find the container with id ec591b3bdb6a69f8c2f202584920b6722fd9896ffe58f4781d17cefe35e8543c
Jul 22 12:46:12.978925 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:12.978856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k" event={"ID":"3277f9b3-452f-4185-847f-5a8fb6e1aec5","Type":"ContainerStarted","Data":"ec591b3bdb6a69f8c2f202584920b6722fd9896ffe58f4781d17cefe35e8543c"}
Jul 22 12:46:14.429577 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.429547    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"]
Jul 22 12:46:14.432604 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.432588    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"
Jul 22 12:46:14.436056 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.436039    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-7scft\""
Jul 22 12:46:14.436249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.436227    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 22 12:46:14.449840 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.449810    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"]
Jul 22 12:46:14.468412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.468389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22vqb\" (UniqueName: \"kubernetes.io/projected/ab974f39-7b3f-43e3-9f09-b1bce0d001c1-kube-api-access-22vqb\") pod \"dns-operator-controller-manager-864dd78c4c-8fhxk\" (UID: \"ab974f39-7b3f-43e3-9f09-b1bce0d001c1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"
Jul 22 12:46:14.569841 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.569716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22vqb\" (UniqueName: \"kubernetes.io/projected/ab974f39-7b3f-43e3-9f09-b1bce0d001c1-kube-api-access-22vqb\") pod \"dns-operator-controller-manager-864dd78c4c-8fhxk\" (UID: \"ab974f39-7b3f-43e3-9f09-b1bce0d001c1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"
Jul 22 12:46:14.587378 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.587347    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22vqb\" (UniqueName: \"kubernetes.io/projected/ab974f39-7b3f-43e3-9f09-b1bce0d001c1-kube-api-access-22vqb\") pod \"dns-operator-controller-manager-864dd78c4c-8fhxk\" (UID: \"ab974f39-7b3f-43e3-9f09-b1bce0d001c1\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"
Jul 22 12:46:14.742813 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.742782    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"
Jul 22 12:46:14.877961 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.877939    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"]
Jul 22 12:46:14.882593 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:46:14.882569    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab974f39_7b3f_43e3_9f09_b1bce0d001c1.slice/crio-4294255c25a635ef5bd9406859d770184057c5ec8a27fd041f1d79cd41047ad1 WatchSource:0}: Error finding container 4294255c25a635ef5bd9406859d770184057c5ec8a27fd041f1d79cd41047ad1: Status 404 returned error can't find the container with id 4294255c25a635ef5bd9406859d770184057c5ec8a27fd041f1d79cd41047ad1
Jul 22 12:46:14.989681 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.989650    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k" event={"ID":"3277f9b3-452f-4185-847f-5a8fb6e1aec5","Type":"ContainerStarted","Data":"b1412c9acf5bfb0a8d9ea8df1f317f8dd8b2fa717e0b618f2db174a4962ae331"}
Jul 22 12:46:14.989869 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.989775    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"
Jul 22 12:46:14.991000 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:14.990979    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk" event={"ID":"ab974f39-7b3f-43e3-9f09-b1bce0d001c1","Type":"ContainerStarted","Data":"4294255c25a635ef5bd9406859d770184057c5ec8a27fd041f1d79cd41047ad1"}
Jul 22 12:46:15.025637 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:15.025589    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k" podStartSLOduration=1.761348632 podStartE2EDuration="4.02557512s" totalImagesPullingTime="2.264226488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:15.02434539 +0000 UTC m=+559.489243439" watchObservedRunningTime="2026-07-22 12:46:15.02557512 +0000 UTC m=+559.490473242"
Jul 22 12:46:18.008643 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.008604    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk" event={"ID":"ab974f39-7b3f-43e3-9f09-b1bce0d001c1","Type":"ContainerStarted","Data":"b52f57a36f97553b0fa52cf6123e86a792a41867c2fc4b6317ff0efec746584b"}
Jul 22 12:46:18.009026 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.008786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"
Jul 22 12:46:18.031054 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.031012    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk" podStartSLOduration=1.50574059 podStartE2EDuration="4.030998896s" totalImagesPullingTime="2.525258306s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:18.029238011 +0000 UTC m=+562.494136059" watchObservedRunningTime="2026-07-22 12:46:18.030998896 +0000 UTC m=+562.495896945"
Jul 22 12:46:18.402720 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.402644    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"]
Jul 22 12:46:18.405229 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.405213    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.407861 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.407843    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gfdhd\""
Jul 22 12:46:18.419432 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.419410    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"]
Jul 22 12:46:18.499937 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.499913    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxk44\" (UniqueName: \"kubernetes.io/projected/4ab86b21-94a0-4096-8e11-3528804adf0b-kube-api-access-wxk44\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.500057 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.499970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ab86b21-94a0-4096-8e11-3528804adf0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.601112 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.601092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ab86b21-94a0-4096-8e11-3528804adf0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.601217 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.601132    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxk44\" (UniqueName: \"kubernetes.io/projected/4ab86b21-94a0-4096-8e11-3528804adf0b-kube-api-access-wxk44\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.601476 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.601457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ab86b21-94a0-4096-8e11-3528804adf0b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.616179 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.616158    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxk44\" (UniqueName: \"kubernetes.io/projected/4ab86b21-94a0-4096-8e11-3528804adf0b-kube-api-access-wxk44\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.714879 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.714854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:18.852012 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:18.851987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"]
Jul 22 12:46:18.856043 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:46:18.856015    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab86b21_94a0_4096_8e11_3528804adf0b.slice/crio-753e6d6b64c19e7e3358c231840da6bb72dcf2550acf53e71a5019bfbf0f1083 WatchSource:0}: Error finding container 753e6d6b64c19e7e3358c231840da6bb72dcf2550acf53e71a5019bfbf0f1083: Status 404 returned error can't find the container with id 753e6d6b64c19e7e3358c231840da6bb72dcf2550acf53e71a5019bfbf0f1083
Jul 22 12:46:19.013599 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:19.013520    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f" event={"ID":"4ab86b21-94a0-4096-8e11-3528804adf0b","Type":"ContainerStarted","Data":"753e6d6b64c19e7e3358c231840da6bb72dcf2550acf53e71a5019bfbf0f1083"}
Jul 22 12:46:20.320017 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.319983    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"]
Jul 22 12:46:20.323152 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.323122    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:20.326570 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.326546    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-6545z\""
Jul 22 12:46:20.336085 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.336063    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"]
Jul 22 12:46:20.416684 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.416656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd5fv\" (UniqueName: \"kubernetes.io/projected/4d159afb-5eba-482d-9c31-fd0f1817f11c-kube-api-access-hd5fv\") pod \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" (UID: \"4d159afb-5eba-482d-9c31-fd0f1817f11c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:20.518007 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.517967    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd5fv\" (UniqueName: \"kubernetes.io/projected/4d159afb-5eba-482d-9c31-fd0f1817f11c-kube-api-access-hd5fv\") pod \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" (UID: \"4d159afb-5eba-482d-9c31-fd0f1817f11c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:20.533057 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.533031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd5fv\" (UniqueName: \"kubernetes.io/projected/4d159afb-5eba-482d-9c31-fd0f1817f11c-kube-api-access-hd5fv\") pod \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" (UID: \"4d159afb-5eba-482d-9c31-fd0f1817f11c\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:20.636070 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.636008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:20.787333 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:20.787307    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"]
Jul 22 12:46:20.790024 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:46:20.789994    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d159afb_5eba_482d_9c31_fd0f1817f11c.slice/crio-d1cb1ce9aa028a419a8ab467c3f76a054a07338c3af003485bb774e069ac8743 WatchSource:0}: Error finding container d1cb1ce9aa028a419a8ab467c3f76a054a07338c3af003485bb774e069ac8743: Status 404 returned error can't find the container with id d1cb1ce9aa028a419a8ab467c3f76a054a07338c3af003485bb774e069ac8743
Jul 22 12:46:21.025705 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:21.025667    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4" event={"ID":"4d159afb-5eba-482d-9c31-fd0f1817f11c","Type":"ContainerStarted","Data":"d1cb1ce9aa028a419a8ab467c3f76a054a07338c3af003485bb774e069ac8743"}
Jul 22 12:46:24.040789 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:24.040741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4" event={"ID":"4d159afb-5eba-482d-9c31-fd0f1817f11c","Type":"ContainerStarted","Data":"04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277"}
Jul 22 12:46:24.041228 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:24.040877    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:24.042573 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:24.042549    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f" event={"ID":"4ab86b21-94a0-4096-8e11-3528804adf0b","Type":"ContainerStarted","Data":"26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0"}
Jul 22 12:46:24.042711 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:24.042694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:24.060932 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:24.060888    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4" podStartSLOduration=1.495468152 podStartE2EDuration="4.060874955s" totalImagesPullingTime="2.565406803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:24.059403359 +0000 UTC m=+568.524301408" watchObservedRunningTime="2026-07-22 12:46:24.060874955 +0000 UTC m=+568.525773055"
Jul 22 12:46:24.079123 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:24.079080    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f" podStartSLOduration=1.56393019 podStartE2EDuration="6.079069274s" totalImagesPullingTime="4.515139084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:24.07811858 +0000 UTC m=+568.543016631" watchObservedRunningTime="2026-07-22 12:46:24.079069274 +0000 UTC m=+568.543967391"
Jul 22 12:46:26.000264 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:26.000226    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mqp5k"
Jul 22 12:46:29.015546 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:29.015512    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-8fhxk"
Jul 22 12:46:35.048628 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:35.048599    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:35.049034 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:35.048666    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:36.816958 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.816930    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"]
Jul 22 12:46:36.817372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.817105    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" containerName="manager" containerID="cri-o://26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0" gracePeriod=2
Jul 22 12:46:36.828399 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.828373    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"]
Jul 22 12:46:36.832543 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:36.832501    2563 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"4ab86b21-94a0-4096-8e11-3528804adf0b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-gt85f\": pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:36.845456 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.845435    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"]
Jul 22 12:46:36.845649 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.845613    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" containerName="manager" containerID="cri-o://04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277" gracePeriod=2
Jul 22 12:46:36.847361 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.847340    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"]
Jul 22 12:46:36.847931 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.847912    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" containerName="manager"
Jul 22 12:46:36.848075 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.848057    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" containerName="manager"
Jul 22 12:46:36.848183 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:36.847932    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:36.848183 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.848172    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" containerName="manager"
Jul 22 12:46:36.851387 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.851367    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"]
Jul 22 12:46:36.851387 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:36.851376    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:36.851561 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.851483    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:36.853915 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:36.853886    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:36.856939 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:36.856910    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:36.872456 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.872417    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"]
Jul 22 12:46:36.885124 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.885099    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"]
Jul 22 12:46:36.885444 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.885429    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" containerName="manager"
Jul 22 12:46:36.885516 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.885449    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" containerName="manager"
Jul 22 12:46:36.885566 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.885520    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" containerName="manager"
Jul 22 12:46:36.888482 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.888407    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"
Jul 22 12:46:36.890833 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:36.890804    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:36.901361 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.901339    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"]
Jul 22 12:46:36.931409 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:36.931384    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:36.942711 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.942691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxbql\" (UniqueName: \"kubernetes.io/projected/351c3933-7613-4aa3-8115-c4b4c559efb8-kube-api-access-qxbql\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jfx9m\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:36.942815 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.942742    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/351c3933-7613-4aa3-8115-c4b4c559efb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jfx9m\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:36.942873 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:36.942854    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7nvz\" (UniqueName: \"kubernetes.io/projected/51bf01fa-5516-42ee-b381-3cea360fd927-kube-api-access-p7nvz\") pod \"limitador-operator-controller-manager-859d7b78bd-mnkcr\" (UID: \"51bf01fa-5516-42ee-b381-3cea360fd927\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"
Jul 22 12:46:37.044339 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.044310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7nvz\" (UniqueName: \"kubernetes.io/projected/51bf01fa-5516-42ee-b381-3cea360fd927-kube-api-access-p7nvz\") pod \"limitador-operator-controller-manager-859d7b78bd-mnkcr\" (UID: \"51bf01fa-5516-42ee-b381-3cea360fd927\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"
Jul 22 12:46:37.044475 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.044395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxbql\" (UniqueName: \"kubernetes.io/projected/351c3933-7613-4aa3-8115-c4b4c559efb8-kube-api-access-qxbql\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jfx9m\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:37.044475 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.044462    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/351c3933-7613-4aa3-8115-c4b4c559efb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jfx9m\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:37.044941 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.044919    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/351c3933-7613-4aa3-8115-c4b4c559efb8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jfx9m\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:37.059674 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.059644    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxbql\" (UniqueName: \"kubernetes.io/projected/351c3933-7613-4aa3-8115-c4b4c559efb8-kube-api-access-qxbql\") pod \"kuadrant-operator-controller-manager-8684c9ff49-jfx9m\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:37.070346 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.070281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7nvz\" (UniqueName: \"kubernetes.io/projected/51bf01fa-5516-42ee-b381-3cea360fd927-kube-api-access-p7nvz\") pod \"limitador-operator-controller-manager-859d7b78bd-mnkcr\" (UID: \"51bf01fa-5516-42ee-b381-3cea360fd927\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"
Jul 22 12:46:37.088526 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.088509    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:37.090930 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.090909    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:37.092972 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.092949    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:37.094631 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.094612    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:37.096101 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.096082    2563 generic.go:356] "Generic (PLEG): container finished" podID="4ab86b21-94a0-4096-8e11-3528804adf0b" containerID="26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0" exitCode=0
Jul 22 12:46:37.096195 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.096128    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:37.096195 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.096170    2563 scope.go:117] "RemoveContainer" containerID="26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0"
Jul 22 12:46:37.097138 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.097035    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:37.097361 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.097333    2563 generic.go:356] "Generic (PLEG): container finished" podID="4d159afb-5eba-482d-9c31-fd0f1817f11c" containerID="04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277" exitCode=0
Jul 22 12:46:37.097428 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.097369    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:37.099114 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.099096    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:37.101045 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.101023    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:37.103852 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.103825    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:37.106831 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.106811    2563 scope.go:117] "RemoveContainer" containerID="26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0"
Jul 22 12:46:37.107093 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.107074    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0\": container with ID starting with 26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0 not found: ID does not exist" containerID="26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0"
Jul 22 12:46:37.107144 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.107101    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0"} err="failed to get container status \"26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0\": rpc error: code = NotFound desc = could not find container \"26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0\": container with ID starting with 26fe7637a0368803395aeb9fdeb973e8d551810414dc56f5ca0966e93812fac0 not found: ID does not exist"
Jul 22 12:46:37.107144 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.107118    2563 scope.go:117] "RemoveContainer" containerID="04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277"
Jul 22 12:46:37.114446 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.114430    2563 scope.go:117] "RemoveContainer" containerID="04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277"
Jul 22 12:46:37.114689 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.114671    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277\": container with ID starting with 04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277 not found: ID does not exist" containerID="04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277"
Jul 22 12:46:37.114742 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.114695    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277"} err="failed to get container status \"04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277\": rpc error: code = NotFound desc = could not find container \"04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277\": container with ID starting with 04ff332c0fe814dae378f0a6c821fa099a8d4e3c1beb20f53c2454b9d2c6c277 not found: ID does not exist"
Jul 22 12:46:37.144706 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.144686    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxk44\" (UniqueName: \"kubernetes.io/projected/4ab86b21-94a0-4096-8e11-3528804adf0b-kube-api-access-wxk44\") pod \"4ab86b21-94a0-4096-8e11-3528804adf0b\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") "
Jul 22 12:46:37.144831 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.144770    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ab86b21-94a0-4096-8e11-3528804adf0b-extensions-socket-volume\") pod \"4ab86b21-94a0-4096-8e11-3528804adf0b\" (UID: \"4ab86b21-94a0-4096-8e11-3528804adf0b\") "
Jul 22 12:46:37.145256 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.145234    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ab86b21-94a0-4096-8e11-3528804adf0b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4ab86b21-94a0-4096-8e11-3528804adf0b" (UID: "4ab86b21-94a0-4096-8e11-3528804adf0b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:37.146715 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.146696    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ab86b21-94a0-4096-8e11-3528804adf0b-kube-api-access-wxk44" (OuterVolumeSpecName: "kube-api-access-wxk44") pod "4ab86b21-94a0-4096-8e11-3528804adf0b" (UID: "4ab86b21-94a0-4096-8e11-3528804adf0b"). InnerVolumeSpecName "kube-api-access-wxk44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:46:37.245237 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.245218    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hd5fv\" (UniqueName: \"kubernetes.io/projected/4d159afb-5eba-482d-9c31-fd0f1817f11c-kube-api-access-hd5fv\") pod \"4d159afb-5eba-482d-9c31-fd0f1817f11c\" (UID: \"4d159afb-5eba-482d-9c31-fd0f1817f11c\") "
Jul 22 12:46:37.245340 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.245321    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxk44\" (UniqueName: \"kubernetes.io/projected/4ab86b21-94a0-4096-8e11-3528804adf0b-kube-api-access-wxk44\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:46:37.245340 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.245332    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4ab86b21-94a0-4096-8e11-3528804adf0b-extensions-socket-volume\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:46:37.247141 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.247120    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d159afb-5eba-482d-9c31-fd0f1817f11c-kube-api-access-hd5fv" (OuterVolumeSpecName: "kube-api-access-hd5fv") pod "4d159afb-5eba-482d-9c31-fd0f1817f11c" (UID: "4d159afb-5eba-482d-9c31-fd0f1817f11c"). InnerVolumeSpecName "kube-api-access-hd5fv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:46:37.272295 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.272273    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:37.278441 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.278426    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"
Jul 22 12:46:37.346313 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.346284    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hd5fv\" (UniqueName: \"kubernetes.io/projected/4d159afb-5eba-482d-9c31-fd0f1817f11c-kube-api-access-hd5fv\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:46:37.415585 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.415555    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:37.418786 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.418739    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:37.420776 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.420731    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-gt85f\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gt85f"
Jul 22 12:46:37.423049 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:37.423019    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-8t9n4\" is forbidden: User \"system:node:ip-10-0-128-197.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-197.ec2.internal' and this object" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8t9n4"
Jul 22 12:46:37.427809 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.427789    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"]
Jul 22 12:46:37.431247 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:46:37.431226    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod351c3933_7613_4aa3_8115_c4b4c559efb8.slice/crio-3ac01bfb89fd448c73e2e98e99aeea7f63c14fb4db9859ba263f78996fcaa6e2 WatchSource:0}: Error finding container 3ac01bfb89fd448c73e2e98e99aeea7f63c14fb4db9859ba263f78996fcaa6e2: Status 404 returned error can't find the container with id 3ac01bfb89fd448c73e2e98e99aeea7f63c14fb4db9859ba263f78996fcaa6e2
Jul 22 12:46:37.445429 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:37.445407    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"]
Jul 22 12:46:37.446521 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:46:37.446502    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51bf01fa_5516_42ee_b381_3cea360fd927.slice/crio-05eda9eac56365761f2413f55fb9072cda1717943cc40a8558b5138d6505645e WatchSource:0}: Error finding container 05eda9eac56365761f2413f55fb9072cda1717943cc40a8558b5138d6505645e: Status 404 returned error can't find the container with id 05eda9eac56365761f2413f55fb9072cda1717943cc40a8558b5138d6505645e
Jul 22 12:46:38.018359 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.018325    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ab86b21-94a0-4096-8e11-3528804adf0b" path="/var/lib/kubelet/pods/4ab86b21-94a0-4096-8e11-3528804adf0b/volumes"
Jul 22 12:46:38.018707 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.018657    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d159afb-5eba-482d-9c31-fd0f1817f11c" path="/var/lib/kubelet/pods/4d159afb-5eba-482d-9c31-fd0f1817f11c/volumes"
Jul 22 12:46:38.102534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.102454    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr" event={"ID":"51bf01fa-5516-42ee-b381-3cea360fd927","Type":"ContainerStarted","Data":"ce28538a238294864391ea13a82de1e659d0191a29ee19a244647bbf827ae6fb"}
Jul 22 12:46:38.102534 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.102509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr" event={"ID":"51bf01fa-5516-42ee-b381-3cea360fd927","Type":"ContainerStarted","Data":"05eda9eac56365761f2413f55fb9072cda1717943cc40a8558b5138d6505645e"}
Jul 22 12:46:38.102735 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.102579    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"
Jul 22 12:46:38.105705 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.105682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m" event={"ID":"351c3933-7613-4aa3-8115-c4b4c559efb8","Type":"ContainerStarted","Data":"30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334"}
Jul 22 12:46:38.105849 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.105724    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m" event={"ID":"351c3933-7613-4aa3-8115-c4b4c559efb8","Type":"ContainerStarted","Data":"3ac01bfb89fd448c73e2e98e99aeea7f63c14fb4db9859ba263f78996fcaa6e2"}
Jul 22 12:46:38.105849 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.105800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:38.140247 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.140205    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr" podStartSLOduration=2.140193556 podStartE2EDuration="2.140193556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:38.13760262 +0000 UTC m=+582.602500669" watchObservedRunningTime="2026-07-22 12:46:38.140193556 +0000 UTC m=+582.605091662"
Jul 22 12:46:38.174528 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:38.174489    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m" podStartSLOduration=2.17447709 podStartE2EDuration="2.17447709s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:46:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:46:38.173523072 +0000 UTC m=+582.638421133" watchObservedRunningTime="2026-07-22 12:46:38.17447709 +0000 UTC m=+582.639375139"
Jul 22 12:46:49.112614 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:49.112583    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mnkcr"
Jul 22 12:46:49.113015 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:49.112764    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:53.425123 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.425091    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"]
Jul 22 12:46:53.425495 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.425253    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m" podUID="351c3933-7613-4aa3-8115-c4b4c559efb8" containerName="manager" containerID="cri-o://30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334" gracePeriod=10
Jul 22 12:46:53.709152 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.709132    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:53.866464 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.866430    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxbql\" (UniqueName: \"kubernetes.io/projected/351c3933-7613-4aa3-8115-c4b4c559efb8-kube-api-access-qxbql\") pod \"351c3933-7613-4aa3-8115-c4b4c559efb8\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") "
Jul 22 12:46:53.866624 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.866509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/351c3933-7613-4aa3-8115-c4b4c559efb8-extensions-socket-volume\") pod \"351c3933-7613-4aa3-8115-c4b4c559efb8\" (UID: \"351c3933-7613-4aa3-8115-c4b4c559efb8\") "
Jul 22 12:46:53.866880 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.866857    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/351c3933-7613-4aa3-8115-c4b4c559efb8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "351c3933-7613-4aa3-8115-c4b4c559efb8" (UID: "351c3933-7613-4aa3-8115-c4b4c559efb8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 12:46:53.868667 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.868638    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/351c3933-7613-4aa3-8115-c4b4c559efb8-kube-api-access-qxbql" (OuterVolumeSpecName: "kube-api-access-qxbql") pod "351c3933-7613-4aa3-8115-c4b4c559efb8" (UID: "351c3933-7613-4aa3-8115-c4b4c559efb8"). InnerVolumeSpecName "kube-api-access-qxbql". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:46:53.967255 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.967232    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxbql\" (UniqueName: \"kubernetes.io/projected/351c3933-7613-4aa3-8115-c4b4c559efb8-kube-api-access-qxbql\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:46:53.967255 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:53.967253    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/351c3933-7613-4aa3-8115-c4b4c559efb8-extensions-socket-volume\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:46:54.178795 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.178762    2563 generic.go:356] "Generic (PLEG): container finished" podID="351c3933-7613-4aa3-8115-c4b4c559efb8" containerID="30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334" exitCode=0
Jul 22 12:46:54.178943 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.178800    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"
Jul 22 12:46:54.178943 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.178836    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m" event={"ID":"351c3933-7613-4aa3-8115-c4b4c559efb8","Type":"ContainerDied","Data":"30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334"}
Jul 22 12:46:54.178943 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.178888    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m" event={"ID":"351c3933-7613-4aa3-8115-c4b4c559efb8","Type":"ContainerDied","Data":"3ac01bfb89fd448c73e2e98e99aeea7f63c14fb4db9859ba263f78996fcaa6e2"}
Jul 22 12:46:54.178943 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.178910    2563 scope.go:117] "RemoveContainer" containerID="30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334"
Jul 22 12:46:54.190737 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.189010    2563 scope.go:117] "RemoveContainer" containerID="30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334"
Jul 22 12:46:54.190737 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:46:54.189300    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334\": container with ID starting with 30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334 not found: ID does not exist" containerID="30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334"
Jul 22 12:46:54.190737 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.189335    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334"} err="failed to get container status \"30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334\": rpc error: code = NotFound desc = could not find container \"30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334\": container with ID starting with 30d001311ec5a676700a9443ed148c90a80d116b92e84a1624717bc77691e334 not found: ID does not exist"
Jul 22 12:46:54.208768 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.208717    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"]
Jul 22 12:46:54.213550 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:54.213527    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-jfx9m"]
Jul 22 12:46:56.017268 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:46:56.017232    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="351c3933-7613-4aa3-8115-c4b4c559efb8" path="/var/lib/kubelet/pods/351c3933-7613-4aa3-8115-c4b4c559efb8/volumes"
Jul 22 12:47:09.814102 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.814073    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"]
Jul 22 12:47:09.814537 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.814363    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="351c3933-7613-4aa3-8115-c4b4c559efb8" containerName="manager"
Jul 22 12:47:09.814537 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.814373    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="351c3933-7613-4aa3-8115-c4b4c559efb8" containerName="manager"
Jul 22 12:47:09.814537 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.814436    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="351c3933-7613-4aa3-8115-c4b4c559efb8" containerName="manager"
Jul 22 12:47:09.817345 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.817322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.819982 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.819955    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-k9xm7\""
Jul 22 12:47:09.835128 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.835106    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"]
Jul 22 12:47:09.986026 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.985998    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986034    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvxfd\" (UniqueName: \"kubernetes.io/projected/d664d98d-27de-4f92-ac66-bc32e06d6209-kube-api-access-tvxfd\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986131    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986370 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986206    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986370 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986227    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986370 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986328    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d664d98d-27de-4f92-ac66-bc32e06d6209-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:09.986467 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:09.986378    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.086935 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.086860    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.086935 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.086895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.086941    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.086968    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087001    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087039    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d664d98d-27de-4f92-ac66-bc32e06d6209-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087084    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087111    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087143 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvxfd\" (UniqueName: \"kubernetes.io/projected/d664d98d-27de-4f92-ac66-bc32e06d6209-kube-api-access-tvxfd\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087471 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087300    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087471 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087367    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087550 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087727 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087699    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.087867 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.087824    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d664d98d-27de-4f92-ac66-bc32e06d6209-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.089443 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.089423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.089563 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.089545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.097413 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.097388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvxfd\" (UniqueName: \"kubernetes.io/projected/d664d98d-27de-4f92-ac66-bc32e06d6209-kube-api-access-tvxfd\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.097493 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.097436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d664d98d-27de-4f92-ac66-bc32e06d6209-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-4jkgx\" (UID: \"d664d98d-27de-4f92-ac66-bc32e06d6209\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.127174 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.127152    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:10.273990 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.273962    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"]
Jul 22 12:47:10.276368 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:47:10.276346    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd664d98d_27de_4f92_ac66_bc32e06d6209.slice/crio-3d8ad42a51b8ad428df7fb46066924c65aaa7ba2e3a6072f626c808953c4bfa6 WatchSource:0}: Error finding container 3d8ad42a51b8ad428df7fb46066924c65aaa7ba2e3a6072f626c808953c4bfa6: Status 404 returned error can't find the container with id 3d8ad42a51b8ad428df7fb46066924c65aaa7ba2e3a6072f626c808953c4bfa6
Jul 22 12:47:10.278242 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.278211    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:47:10.278322 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.278280    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:47:10.278322 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:10.278307    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:47:11.250029 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:11.249996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" event={"ID":"d664d98d-27de-4f92-ac66-bc32e06d6209","Type":"ContainerStarted","Data":"25756890d13681a053f47b66979024f0828919030947331370464fea8c59d600"}
Jul 22 12:47:11.250390 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:11.250048    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" event={"ID":"d664d98d-27de-4f92-ac66-bc32e06d6209","Type":"ContainerStarted","Data":"3d8ad42a51b8ad428df7fb46066924c65aaa7ba2e3a6072f626c808953c4bfa6"}
Jul 22 12:47:11.274792 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:11.274729    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" podStartSLOduration=2.274714018 podStartE2EDuration="2.274714018s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:47:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:47:11.271866114 +0000 UTC m=+615.736764184" watchObservedRunningTime="2026-07-22 12:47:11.274714018 +0000 UTC m=+615.739612067"
Jul 22 12:47:12.127811 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.127778    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:12.132232 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.132209    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:12.254004 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.253977    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:12.254912 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.254894    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:47:12.373207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.373176    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-68c87"]
Jul 22 12:47:12.375389 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.375374    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.377889 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.377834    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 22 12:47:12.377889 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.377883    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-d77zv\""
Jul 22 12:47:12.389465 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.389443    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-68c87"]
Jul 22 12:47:12.463510 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.463482    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-68c87"]
Jul 22 12:47:12.506893 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.506871    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/92d9faeb-4adb-4658-92e9-589be2112144-config-file\") pod \"limitador-limitador-64c94fd64f-68c87\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.507038 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.506905    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xb7v\" (UniqueName: \"kubernetes.io/projected/92d9faeb-4adb-4658-92e9-589be2112144-kube-api-access-9xb7v\") pod \"limitador-limitador-64c94fd64f-68c87\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.608249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.608214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/92d9faeb-4adb-4658-92e9-589be2112144-config-file\") pod \"limitador-limitador-64c94fd64f-68c87\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.608249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.608256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xb7v\" (UniqueName: \"kubernetes.io/projected/92d9faeb-4adb-4658-92e9-589be2112144-kube-api-access-9xb7v\") pod \"limitador-limitador-64c94fd64f-68c87\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.608911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.608893    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/92d9faeb-4adb-4658-92e9-589be2112144-config-file\") pod \"limitador-limitador-64c94fd64f-68c87\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.621933 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.621910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xb7v\" (UniqueName: \"kubernetes.io/projected/92d9faeb-4adb-4658-92e9-589be2112144-kube-api-access-9xb7v\") pod \"limitador-limitador-64c94fd64f-68c87\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.685741 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.685681    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:12.844802 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:12.844782    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-68c87"]
Jul 22 12:47:12.847522 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:47:12.847489    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92d9faeb_4adb_4658_92e9_589be2112144.slice/crio-9d0889ab4db7efe38367f397b567f739d24f46d4d97d6b7fc41505b796e46faa WatchSource:0}: Error finding container 9d0889ab4db7efe38367f397b567f739d24f46d4d97d6b7fc41505b796e46faa: Status 404 returned error can't find the container with id 9d0889ab4db7efe38367f397b567f739d24f46d4d97d6b7fc41505b796e46faa
Jul 22 12:47:13.227104 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.227073    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-5s5kx"]
Jul 22 12:47:13.229788 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.229770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5s5kx"
Jul 22 12:47:13.232396 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.232373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9qhxh\""
Jul 22 12:47:13.244603 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.244581    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5s5kx"]
Jul 22 12:47:13.259961 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.259937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87" event={"ID":"92d9faeb-4adb-4658-92e9-589be2112144","Type":"ContainerStarted","Data":"9d0889ab4db7efe38367f397b567f739d24f46d4d97d6b7fc41505b796e46faa"}
Jul 22 12:47:13.416007 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.415971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7p24\" (UniqueName: \"kubernetes.io/projected/72b74e6d-707d-41a0-b29f-5f535912d24d-kube-api-access-b7p24\") pod \"authorino-58cd955675-5s5kx\" (UID: \"72b74e6d-707d-41a0-b29f-5f535912d24d\") " pod="kuadrant-system/authorino-58cd955675-5s5kx"
Jul 22 12:47:13.516791 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.516710    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7p24\" (UniqueName: \"kubernetes.io/projected/72b74e6d-707d-41a0-b29f-5f535912d24d-kube-api-access-b7p24\") pod \"authorino-58cd955675-5s5kx\" (UID: \"72b74e6d-707d-41a0-b29f-5f535912d24d\") " pod="kuadrant-system/authorino-58cd955675-5s5kx"
Jul 22 12:47:13.527105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.527075    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7p24\" (UniqueName: \"kubernetes.io/projected/72b74e6d-707d-41a0-b29f-5f535912d24d-kube-api-access-b7p24\") pod \"authorino-58cd955675-5s5kx\" (UID: \"72b74e6d-707d-41a0-b29f-5f535912d24d\") " pod="kuadrant-system/authorino-58cd955675-5s5kx"
Jul 22 12:47:13.538830 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.538812    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5s5kx"
Jul 22 12:47:13.667448 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:13.667373    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-5s5kx"]
Jul 22 12:47:13.669653 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:47:13.669627    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72b74e6d_707d_41a0_b29f_5f535912d24d.slice/crio-44360d82eae930259031226f04dd5142d730ea83955d08b2553c1c059e28691d WatchSource:0}: Error finding container 44360d82eae930259031226f04dd5142d730ea83955d08b2553c1c059e28691d: Status 404 returned error can't find the container with id 44360d82eae930259031226f04dd5142d730ea83955d08b2553c1c059e28691d
Jul 22 12:47:14.266733 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:14.266695    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5s5kx" event={"ID":"72b74e6d-707d-41a0-b29f-5f535912d24d","Type":"ContainerStarted","Data":"44360d82eae930259031226f04dd5142d730ea83955d08b2553c1c059e28691d"}
Jul 22 12:47:18.290287 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:18.290251    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87" event={"ID":"92d9faeb-4adb-4658-92e9-589be2112144","Type":"ContainerStarted","Data":"ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79"}
Jul 22 12:47:18.290802 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:18.290322    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:18.291555 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:18.291533    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5s5kx" event={"ID":"72b74e6d-707d-41a0-b29f-5f535912d24d","Type":"ContainerStarted","Data":"0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500"}
Jul 22 12:47:18.317942 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:18.317895    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87" podStartSLOduration=1.474125648 podStartE2EDuration="6.317880179s" totalImagesPullingTime="4.843754531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:47:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:47:18.317465135 +0000 UTC m=+622.782363181" watchObservedRunningTime="2026-07-22 12:47:18.317880179 +0000 UTC m=+622.782778231"
Jul 22 12:47:18.346006 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:18.345958    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-5s5kx" podStartSLOduration=1.322744092 podStartE2EDuration="5.345942583s" totalImagesPullingTime="4.023198491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:47:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:47:18.344707797 +0000 UTC m=+622.809605857" watchObservedRunningTime="2026-07-22 12:47:18.345942583 +0000 UTC m=+622.810840637"
Jul 22 12:47:20.011243 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:20.011215    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5s5kx"]
Jul 22 12:47:20.299980 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:20.299897    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-5s5kx" podUID="72b74e6d-707d-41a0-b29f-5f535912d24d" containerName="authorino" containerID="cri-o://0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500" gracePeriod=30
Jul 22 12:47:20.538964 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:20.538942    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5s5kx"
Jul 22 12:47:20.577451 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:20.577386    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7p24\" (UniqueName: \"kubernetes.io/projected/72b74e6d-707d-41a0-b29f-5f535912d24d-kube-api-access-b7p24\") pod \"72b74e6d-707d-41a0-b29f-5f535912d24d\" (UID: \"72b74e6d-707d-41a0-b29f-5f535912d24d\") "
Jul 22 12:47:20.579446 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:20.579417    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72b74e6d-707d-41a0-b29f-5f535912d24d-kube-api-access-b7p24" (OuterVolumeSpecName: "kube-api-access-b7p24") pod "72b74e6d-707d-41a0-b29f-5f535912d24d" (UID: "72b74e6d-707d-41a0-b29f-5f535912d24d"). InnerVolumeSpecName "kube-api-access-b7p24". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:47:20.678707 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:20.678678    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7p24\" (UniqueName: \"kubernetes.io/projected/72b74e6d-707d-41a0-b29f-5f535912d24d-kube-api-access-b7p24\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:47:21.305181 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.305143    2563 generic.go:356] "Generic (PLEG): container finished" podID="72b74e6d-707d-41a0-b29f-5f535912d24d" containerID="0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500" exitCode=0
Jul 22 12:47:21.305645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.305202    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5s5kx" event={"ID":"72b74e6d-707d-41a0-b29f-5f535912d24d","Type":"ContainerDied","Data":"0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500"}
Jul 22 12:47:21.305645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.305204    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-5s5kx"
Jul 22 12:47:21.305645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.305232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-5s5kx" event={"ID":"72b74e6d-707d-41a0-b29f-5f535912d24d","Type":"ContainerDied","Data":"44360d82eae930259031226f04dd5142d730ea83955d08b2553c1c059e28691d"}
Jul 22 12:47:21.305645 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.305279    2563 scope.go:117] "RemoveContainer" containerID="0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500"
Jul 22 12:47:21.313714 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.313695    2563 scope.go:117] "RemoveContainer" containerID="0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500"
Jul 22 12:47:21.313983 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:47:21.313966    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500\": container with ID starting with 0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500 not found: ID does not exist" containerID="0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500"
Jul 22 12:47:21.314071 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.313990    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500"} err="failed to get container status \"0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500\": rpc error: code = NotFound desc = could not find container \"0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500\": container with ID starting with 0bfb56495a4b2f39cd7f8d26ca8dd986df1e2505a247a3eab29ee9a6f0824500 not found: ID does not exist"
Jul 22 12:47:21.327286 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.327266    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-5s5kx"]
Jul 22 12:47:21.331386 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:21.331368    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-5s5kx"]
Jul 22 12:47:22.017458 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:22.017426    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72b74e6d-707d-41a0-b29f-5f535912d24d" path="/var/lib/kubelet/pods/72b74e6d-707d-41a0-b29f-5f535912d24d/volumes"
Jul 22 12:47:29.297105 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:29.297077    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:31.014341 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.014306    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-68c87"]
Jul 22 12:47:31.014719 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.014489    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87" podUID="92d9faeb-4adb-4658-92e9-589be2112144" containerName="limitador" containerID="cri-o://ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79" gracePeriod=30
Jul 22 12:47:31.564990 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.564967    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:31.663300 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.663239    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/92d9faeb-4adb-4658-92e9-589be2112144-config-file\") pod \"92d9faeb-4adb-4658-92e9-589be2112144\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") "
Jul 22 12:47:31.663428 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.663306    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xb7v\" (UniqueName: \"kubernetes.io/projected/92d9faeb-4adb-4658-92e9-589be2112144-kube-api-access-9xb7v\") pod \"92d9faeb-4adb-4658-92e9-589be2112144\" (UID: \"92d9faeb-4adb-4658-92e9-589be2112144\") "
Jul 22 12:47:31.663595 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.663572    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92d9faeb-4adb-4658-92e9-589be2112144-config-file" (OuterVolumeSpecName: "config-file") pod "92d9faeb-4adb-4658-92e9-589be2112144" (UID: "92d9faeb-4adb-4658-92e9-589be2112144"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 12:47:31.665385 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.665365    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92d9faeb-4adb-4658-92e9-589be2112144-kube-api-access-9xb7v" (OuterVolumeSpecName: "kube-api-access-9xb7v") pod "92d9faeb-4adb-4658-92e9-589be2112144" (UID: "92d9faeb-4adb-4658-92e9-589be2112144"). InnerVolumeSpecName "kube-api-access-9xb7v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:47:31.763953 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.763926    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xb7v\" (UniqueName: \"kubernetes.io/projected/92d9faeb-4adb-4658-92e9-589be2112144-kube-api-access-9xb7v\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:47:31.763953 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:31.763951    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/92d9faeb-4adb-4658-92e9-589be2112144-config-file\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:47:32.353332 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.353258    2563 generic.go:356] "Generic (PLEG): container finished" podID="92d9faeb-4adb-4658-92e9-589be2112144" containerID="ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79" exitCode=0
Jul 22 12:47:32.353332 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.353302    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87"
Jul 22 12:47:32.353790 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.353339    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87" event={"ID":"92d9faeb-4adb-4658-92e9-589be2112144","Type":"ContainerDied","Data":"ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79"}
Jul 22 12:47:32.353790 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.353388    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-68c87" event={"ID":"92d9faeb-4adb-4658-92e9-589be2112144","Type":"ContainerDied","Data":"9d0889ab4db7efe38367f397b567f739d24f46d4d97d6b7fc41505b796e46faa"}
Jul 22 12:47:32.353790 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.353410    2563 scope.go:117] "RemoveContainer" containerID="ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79"
Jul 22 12:47:32.361785 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.361769    2563 scope.go:117] "RemoveContainer" containerID="ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79"
Jul 22 12:47:32.362035 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:47:32.362020    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79\": container with ID starting with ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79 not found: ID does not exist" containerID="ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79"
Jul 22 12:47:32.362082 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.362044    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79"} err="failed to get container status \"ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79\": rpc error: code = NotFound desc = could not find container \"ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79\": container with ID starting with ab543268aece0f49b41da3523f065794115ecc58c177313197cd26eacb632b79 not found: ID does not exist"
Jul 22 12:47:32.370066 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.370045    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-68c87"]
Jul 22 12:47:32.373886 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:32.373863    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-68c87"]
Jul 22 12:47:34.018545 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.018508    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92d9faeb-4adb-4658-92e9-589be2112144" path="/var/lib/kubelet/pods/92d9faeb-4adb-4658-92e9-589be2112144/volumes"
Jul 22 12:47:34.018915 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.018868    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"]
Jul 22 12:47:34.019305 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.019284    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72b74e6d-707d-41a0-b29f-5f535912d24d" containerName="authorino"
Jul 22 12:47:34.019463 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.019444    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="72b74e6d-707d-41a0-b29f-5f535912d24d" containerName="authorino"
Jul 22 12:47:34.019548 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.019494    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92d9faeb-4adb-4658-92e9-589be2112144" containerName="limitador"
Jul 22 12:47:34.019548 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.019503    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="92d9faeb-4adb-4658-92e9-589be2112144" containerName="limitador"
Jul 22 12:47:34.019614 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.019580    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="92d9faeb-4adb-4658-92e9-589be2112144" containerName="limitador"
Jul 22 12:47:34.019614 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.019592    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="72b74e6d-707d-41a0-b29f-5f535912d24d" containerName="authorino"
Jul 22 12:47:34.023953 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.023938    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.028128 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.028105    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-cccrg\""
Jul 22 12:47:34.028206 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.028160    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 22 12:47:34.029199 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.029182    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 22 12:47:34.029294 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.029231    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 22 12:47:34.034003 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.033985    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"]
Jul 22 12:47:34.082650 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.082622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvcrz\" (UniqueName: \"kubernetes.io/projected/ce0de7c3-21db-4b3c-8612-62857d1130a3-kube-api-access-wvcrz\") pod \"postgres-69fb4594cf-ljwq7\" (UID: \"ce0de7c3-21db-4b3c-8612-62857d1130a3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.082778 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.082703    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ce0de7c3-21db-4b3c-8612-62857d1130a3-data\") pod \"postgres-69fb4594cf-ljwq7\" (UID: \"ce0de7c3-21db-4b3c-8612-62857d1130a3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.183810 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.183774    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ce0de7c3-21db-4b3c-8612-62857d1130a3-data\") pod \"postgres-69fb4594cf-ljwq7\" (UID: \"ce0de7c3-21db-4b3c-8612-62857d1130a3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.183959 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.183848    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvcrz\" (UniqueName: \"kubernetes.io/projected/ce0de7c3-21db-4b3c-8612-62857d1130a3-kube-api-access-wvcrz\") pod \"postgres-69fb4594cf-ljwq7\" (UID: \"ce0de7c3-21db-4b3c-8612-62857d1130a3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.184128 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.184110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/ce0de7c3-21db-4b3c-8612-62857d1130a3-data\") pod \"postgres-69fb4594cf-ljwq7\" (UID: \"ce0de7c3-21db-4b3c-8612-62857d1130a3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.194285 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.194256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvcrz\" (UniqueName: \"kubernetes.io/projected/ce0de7c3-21db-4b3c-8612-62857d1130a3-kube-api-access-wvcrz\") pod \"postgres-69fb4594cf-ljwq7\" (UID: \"ce0de7c3-21db-4b3c-8612-62857d1130a3\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.334076 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.334007    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:34.465739 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:34.465713    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"]
Jul 22 12:47:34.469185 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:47:34.469159    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0de7c3_21db_4b3c_8612_62857d1130a3.slice/crio-dc07d9ab99add7f7fb18ee7ac8fefff4d1aab796f69f9fa569a40e6eeacc574f WatchSource:0}: Error finding container dc07d9ab99add7f7fb18ee7ac8fefff4d1aab796f69f9fa569a40e6eeacc574f: Status 404 returned error can't find the container with id dc07d9ab99add7f7fb18ee7ac8fefff4d1aab796f69f9fa569a40e6eeacc574f
Jul 22 12:47:35.372726 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:35.372654    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7" event={"ID":"ce0de7c3-21db-4b3c-8612-62857d1130a3","Type":"ContainerStarted","Data":"dc07d9ab99add7f7fb18ee7ac8fefff4d1aab796f69f9fa569a40e6eeacc574f"}
Jul 22 12:47:39.395449 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:39.395415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7" event={"ID":"ce0de7c3-21db-4b3c-8612-62857d1130a3","Type":"ContainerStarted","Data":"ca81a07bff3512757bc03a8755425fc4c8108bdd452c00aa8965497d550caa36"}
Jul 22 12:47:39.395893 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:39.395490    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:39.414943 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:39.414899    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7" podStartSLOduration=1.6700908559999998 podStartE2EDuration="6.414884881s" totalImagesPullingTime="4.744794025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:47:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:47:39.414202289 +0000 UTC m=+643.879100340" watchObservedRunningTime="2026-07-22 12:47:39.414884881 +0000 UTC m=+643.879782929"
Jul 22 12:47:45.430429 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:45.430400    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-ljwq7"
Jul 22 12:47:45.984818 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:45.984784    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gndtz"]
Jul 22 12:47:45.988055 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:45.988036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gndtz"
Jul 22 12:47:45.990454 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:45.990434    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-9qhxh\""
Jul 22 12:47:45.995307 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:45.995286    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gndtz"]
Jul 22 12:47:46.081870 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.081844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2csn\" (UniqueName: \"kubernetes.io/projected/2398e9f5-9317-45fe-a886-87b4a8299051-kube-api-access-v2csn\") pod \"authorino-5d4b67c446-gndtz\" (UID: \"2398e9f5-9317-45fe-a886-87b4a8299051\") " pod="kuadrant-system/authorino-5d4b67c446-gndtz"
Jul 22 12:47:46.182565 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.182540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2csn\" (UniqueName: \"kubernetes.io/projected/2398e9f5-9317-45fe-a886-87b4a8299051-kube-api-access-v2csn\") pod \"authorino-5d4b67c446-gndtz\" (UID: \"2398e9f5-9317-45fe-a886-87b4a8299051\") " pod="kuadrant-system/authorino-5d4b67c446-gndtz"
Jul 22 12:47:46.190371 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.190350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2csn\" (UniqueName: \"kubernetes.io/projected/2398e9f5-9317-45fe-a886-87b4a8299051-kube-api-access-v2csn\") pod \"authorino-5d4b67c446-gndtz\" (UID: \"2398e9f5-9317-45fe-a886-87b4a8299051\") " pod="kuadrant-system/authorino-5d4b67c446-gndtz"
Jul 22 12:47:46.214578 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.214557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gndtz"]
Jul 22 12:47:46.214785 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.214772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gndtz"
Jul 22 12:47:46.241659 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.241608    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7db7cdf646-8bxd6"]
Jul 22 12:47:46.246369 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.246351    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7cdf646-8bxd6"
Jul 22 12:47:46.254184 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.253905    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7db7cdf646-8bxd6"]
Jul 22 12:47:46.352579 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.352557    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gndtz"]
Jul 22 12:47:46.355385 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:47:46.355360    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2398e9f5_9317_45fe_a886_87b4a8299051.slice/crio-bf415c51fbdb2dd395abdab3bf2f3285a00b8370386f7b864ba0e5bfc9fd76a0 WatchSource:0}: Error finding container bf415c51fbdb2dd395abdab3bf2f3285a00b8370386f7b864ba0e5bfc9fd76a0: Status 404 returned error can't find the container with id bf415c51fbdb2dd395abdab3bf2f3285a00b8370386f7b864ba0e5bfc9fd76a0
Jul 22 12:47:46.383695 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.383665    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhsvk\" (UniqueName: \"kubernetes.io/projected/e56dcb69-3a4d-4253-990b-ca30ff32e5b2-kube-api-access-zhsvk\") pod \"authorino-7db7cdf646-8bxd6\" (UID: \"e56dcb69-3a4d-4253-990b-ca30ff32e5b2\") " pod="kuadrant-system/authorino-7db7cdf646-8bxd6"
Jul 22 12:47:46.427064 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.427038    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gndtz" event={"ID":"2398e9f5-9317-45fe-a886-87b4a8299051","Type":"ContainerStarted","Data":"bf415c51fbdb2dd395abdab3bf2f3285a00b8370386f7b864ba0e5bfc9fd76a0"}
Jul 22 12:47:46.476033 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.476006    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7db7cdf646-8bxd6"]
Jul 22 12:47:46.476312 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:47:46.476214    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-zhsvk], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-7db7cdf646-8bxd6" podUID="e56dcb69-3a4d-4253-990b-ca30ff32e5b2"
Jul 22 12:47:46.484531 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.484506    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhsvk\" (UniqueName: \"kubernetes.io/projected/e56dcb69-3a4d-4253-990b-ca30ff32e5b2-kube-api-access-zhsvk\") pod \"authorino-7db7cdf646-8bxd6\" (UID: \"e56dcb69-3a4d-4253-990b-ca30ff32e5b2\") " pod="kuadrant-system/authorino-7db7cdf646-8bxd6"
Jul 22 12:47:46.499712 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.499666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhsvk\" (UniqueName: \"kubernetes.io/projected/e56dcb69-3a4d-4253-990b-ca30ff32e5b2-kube-api-access-zhsvk\") pod \"authorino-7db7cdf646-8bxd6\" (UID: \"e56dcb69-3a4d-4253-990b-ca30ff32e5b2\") " pod="kuadrant-system/authorino-7db7cdf646-8bxd6"
Jul 22 12:47:46.507282 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.507264    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8667994f5d-gfmkr"]
Jul 22 12:47:46.518766 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.518729    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8667994f5d-gfmkr"]
Jul 22 12:47:46.518853 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.518831    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.521383 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.521362    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 22 12:47:46.585838 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.585815    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjwbt\" (UniqueName: \"kubernetes.io/projected/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-kube-api-access-pjwbt\") pod \"authorino-8667994f5d-gfmkr\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") " pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.585925 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.585862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-tls-cert\") pod \"authorino-8667994f5d-gfmkr\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") " pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.686707 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.686671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjwbt\" (UniqueName: \"kubernetes.io/projected/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-kube-api-access-pjwbt\") pod \"authorino-8667994f5d-gfmkr\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") " pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.686818 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.686724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-tls-cert\") pod \"authorino-8667994f5d-gfmkr\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") " pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.689100 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.689083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-tls-cert\") pod \"authorino-8667994f5d-gfmkr\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") " pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.694437 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.694421    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjwbt\" (UniqueName: \"kubernetes.io/projected/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-kube-api-access-pjwbt\") pod \"authorino-8667994f5d-gfmkr\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") " pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.829350 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.829298    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:47:46.975177 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:46.975064    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8667994f5d-gfmkr"]
Jul 22 12:47:46.977232 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:47:46.977204    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc8258fe8_c06a_44fd_b801_31fe2ab15c4f.slice/crio-1b3297dfbfe89adb47680bdc0b1f4646c1dce8205bc6a4172195458b52d81a84 WatchSource:0}: Error finding container 1b3297dfbfe89adb47680bdc0b1f4646c1dce8205bc6a4172195458b52d81a84: Status 404 returned error can't find the container with id 1b3297dfbfe89adb47680bdc0b1f4646c1dce8205bc6a4172195458b52d81a84
Jul 22 12:47:47.432456 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.432370    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gndtz" event={"ID":"2398e9f5-9317-45fe-a886-87b4a8299051","Type":"ContainerStarted","Data":"8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8"}
Jul 22 12:47:47.432650 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.432444    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-gndtz" podUID="2398e9f5-9317-45fe-a886-87b4a8299051" containerName="authorino" containerID="cri-o://8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8" gracePeriod=30
Jul 22 12:47:47.433990 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.433959    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8667994f5d-gfmkr" event={"ID":"c8258fe8-c06a-44fd-b801-31fe2ab15c4f","Type":"ContainerStarted","Data":"1b3297dfbfe89adb47680bdc0b1f4646c1dce8205bc6a4172195458b52d81a84"}
Jul 22 12:47:47.433990 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.433968    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7cdf646-8bxd6"
Jul 22 12:47:47.448659 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.448618    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-gndtz" podStartSLOduration=1.7546321919999999 podStartE2EDuration="2.44860573s" totalImagesPullingTime="693.973538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:47:47.446975729 +0000 UTC m=+651.911873778" watchObservedRunningTime="2026-07-22 12:47:47.44860573 +0000 UTC m=+651.913503779"
Jul 22 12:47:47.461316 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.461295    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7cdf646-8bxd6"
Jul 22 12:47:47.594149 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.593945    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhsvk\" (UniqueName: \"kubernetes.io/projected/e56dcb69-3a4d-4253-990b-ca30ff32e5b2-kube-api-access-zhsvk\") pod \"e56dcb69-3a4d-4253-990b-ca30ff32e5b2\" (UID: \"e56dcb69-3a4d-4253-990b-ca30ff32e5b2\") "
Jul 22 12:47:47.605831 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.605790    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56dcb69-3a4d-4253-990b-ca30ff32e5b2-kube-api-access-zhsvk" (OuterVolumeSpecName: "kube-api-access-zhsvk") pod "e56dcb69-3a4d-4253-990b-ca30ff32e5b2" (UID: "e56dcb69-3a4d-4253-990b-ca30ff32e5b2"). InnerVolumeSpecName "kube-api-access-zhsvk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:47:47.696792 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.696737    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhsvk\" (UniqueName: \"kubernetes.io/projected/e56dcb69-3a4d-4253-990b-ca30ff32e5b2-kube-api-access-zhsvk\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:47:47.707295 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.707271    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gndtz"
Jul 22 12:47:47.798080 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.798054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2csn\" (UniqueName: \"kubernetes.io/projected/2398e9f5-9317-45fe-a886-87b4a8299051-kube-api-access-v2csn\") pod \"2398e9f5-9317-45fe-a886-87b4a8299051\" (UID: \"2398e9f5-9317-45fe-a886-87b4a8299051\") "
Jul 22 12:47:47.800218 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.800192    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2398e9f5-9317-45fe-a886-87b4a8299051-kube-api-access-v2csn" (OuterVolumeSpecName: "kube-api-access-v2csn") pod "2398e9f5-9317-45fe-a886-87b4a8299051" (UID: "2398e9f5-9317-45fe-a886-87b4a8299051"). InnerVolumeSpecName "kube-api-access-v2csn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:47:47.900880 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:47.900855    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2csn\" (UniqueName: \"kubernetes.io/projected/2398e9f5-9317-45fe-a886-87b4a8299051-kube-api-access-v2csn\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:47:48.440415 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.440386    2563 generic.go:356] "Generic (PLEG): container finished" podID="2398e9f5-9317-45fe-a886-87b4a8299051" containerID="8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8" exitCode=0
Jul 22 12:47:48.440599 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.440446    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-gndtz"
Jul 22 12:47:48.440599 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.440462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gndtz" event={"ID":"2398e9f5-9317-45fe-a886-87b4a8299051","Type":"ContainerDied","Data":"8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8"}
Jul 22 12:47:48.440599 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.440510    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-gndtz" event={"ID":"2398e9f5-9317-45fe-a886-87b4a8299051","Type":"ContainerDied","Data":"bf415c51fbdb2dd395abdab3bf2f3285a00b8370386f7b864ba0e5bfc9fd76a0"}
Jul 22 12:47:48.440599 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.440527    2563 scope.go:117] "RemoveContainer" containerID="8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8"
Jul 22 12:47:48.442219 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.442180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8667994f5d-gfmkr" event={"ID":"c8258fe8-c06a-44fd-b801-31fe2ab15c4f","Type":"ContainerStarted","Data":"7b63598c1abbb9225c2647f98e03e3088e38bc9d6e59583cb540e3bef954182c"}
Jul 22 12:47:48.442331 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.442216    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db7cdf646-8bxd6"
Jul 22 12:47:48.449058 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.449041    2563 scope.go:117] "RemoveContainer" containerID="8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8"
Jul 22 12:47:48.449317 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:47:48.449288    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8\": container with ID starting with 8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8 not found: ID does not exist" containerID="8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8"
Jul 22 12:47:48.449404 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.449322    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8"} err="failed to get container status \"8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8\": rpc error: code = NotFound desc = could not find container \"8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8\": container with ID starting with 8dc0d7f1f311fd7ae8073c8d56fdeb5e7fc19ed38431b1f009631b4436e347c8 not found: ID does not exist"
Jul 22 12:47:48.457470 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.457449    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gndtz"]
Jul 22 12:47:48.463500 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.463480    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-gndtz"]
Jul 22 12:47:48.478574 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.478537    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-8667994f5d-gfmkr" podStartSLOduration=1.9031756770000001 podStartE2EDuration="2.478526212s" totalImagesPullingTime="575.350535ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:47:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:47:48.477636673 +0000 UTC m=+652.942534724" watchObservedRunningTime="2026-07-22 12:47:48.478526212 +0000 UTC m=+652.943424269"
Jul 22 12:47:48.503883 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.503854    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7db7cdf646-8bxd6"]
Jul 22 12:47:48.526594 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:48.526568    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7db7cdf646-8bxd6"]
Jul 22 12:47:50.017923 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:50.017892    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2398e9f5-9317-45fe-a886-87b4a8299051" path="/var/lib/kubelet/pods/2398e9f5-9317-45fe-a886-87b4a8299051/volumes"
Jul 22 12:47:50.018283 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:50.018199    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e56dcb69-3a4d-4253-990b-ca30ff32e5b2" path="/var/lib/kubelet/pods/e56dcb69-3a4d-4253-990b-ca30ff32e5b2/volumes"
Jul 22 12:47:53.274554 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.274523    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fbb74bd86-5dbsz"]
Jul 22 12:47:53.275018 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.274879    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2398e9f5-9317-45fe-a886-87b4a8299051" containerName="authorino"
Jul 22 12:47:53.275018 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.274890    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2398e9f5-9317-45fe-a886-87b4a8299051" containerName="authorino"
Jul 22 12:47:53.275018 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.274941    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2398e9f5-9317-45fe-a886-87b4a8299051" containerName="authorino"
Jul 22 12:47:53.284106 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.284084    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.287039 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.287018    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 22 12:47:53.287982 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.287959    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-ns5lk\""
Jul 22 12:47:53.287982 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.287980    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 12:47:53.289099 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.289073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fbb74bd86-5dbsz"]
Jul 22 12:47:53.440518 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.440485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/961602ee-dd37-4363-b81b-1a7f24ca17cf-webhook-cert\") pod \"maas-controller-7fbb74bd86-5dbsz\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") " pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.440707 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.440584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5qtk\" (UniqueName: \"kubernetes.io/projected/961602ee-dd37-4363-b81b-1a7f24ca17cf-kube-api-access-r5qtk\") pod \"maas-controller-7fbb74bd86-5dbsz\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") " pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.541940 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.541863    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5qtk\" (UniqueName: \"kubernetes.io/projected/961602ee-dd37-4363-b81b-1a7f24ca17cf-kube-api-access-r5qtk\") pod \"maas-controller-7fbb74bd86-5dbsz\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") " pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.541940 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.541932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/961602ee-dd37-4363-b81b-1a7f24ca17cf-webhook-cert\") pod \"maas-controller-7fbb74bd86-5dbsz\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") " pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.544585 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.544553    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/961602ee-dd37-4363-b81b-1a7f24ca17cf-webhook-cert\") pod \"maas-controller-7fbb74bd86-5dbsz\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") " pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.551068 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.551044    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5qtk\" (UniqueName: \"kubernetes.io/projected/961602ee-dd37-4363-b81b-1a7f24ca17cf-kube-api-access-r5qtk\") pod \"maas-controller-7fbb74bd86-5dbsz\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") " pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.595474 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.595452    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:47:53.727491 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:53.727467    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fbb74bd86-5dbsz"]
Jul 22 12:47:53.730132 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:47:53.730104    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod961602ee_dd37_4363_b81b_1a7f24ca17cf.slice/crio-e3b93800dc8610b99eee2dc3ce30154d1e1f0ee2231f08aebe8bb88444dbf93c WatchSource:0}: Error finding container e3b93800dc8610b99eee2dc3ce30154d1e1f0ee2231f08aebe8bb88444dbf93c: Status 404 returned error can't find the container with id e3b93800dc8610b99eee2dc3ce30154d1e1f0ee2231f08aebe8bb88444dbf93c
Jul 22 12:47:54.472220 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:54.472181    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz" event={"ID":"961602ee-dd37-4363-b81b-1a7f24ca17cf","Type":"ContainerStarted","Data":"e3b93800dc8610b99eee2dc3ce30154d1e1f0ee2231f08aebe8bb88444dbf93c"}
Jul 22 12:47:55.976099 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:55.976075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 22 12:47:56.483962 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:56.483930    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz" event={"ID":"961602ee-dd37-4363-b81b-1a7f24ca17cf","Type":"ContainerStarted","Data":"ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3"}
Jul 22 12:47:56.484144 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:47:56.484055    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:48:07.490067 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:07.490037    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:48:07.512193 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:07.512145    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz" podStartSLOduration=12.270507687 podStartE2EDuration="14.512131377s" totalImagesPullingTime="2.24162369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:47:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:47:56.507379363 +0000 UTC m=+660.972277572" watchObservedRunningTime="2026-07-22 12:48:07.512131377 +0000 UTC m=+671.977029426"
Jul 22 12:48:07.749163 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:07.749085    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fbb74bd86-5dbsz"]
Jul 22 12:48:07.749302 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:07.749275    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz" podUID="961602ee-dd37-4363-b81b-1a7f24ca17cf" containerName="manager" containerID="cri-o://ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3" gracePeriod=30
Jul 22 12:48:07.989662 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:07.989639    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:48:08.164738 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.164662    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/961602ee-dd37-4363-b81b-1a7f24ca17cf-webhook-cert\") pod \"961602ee-dd37-4363-b81b-1a7f24ca17cf\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") "
Jul 22 12:48:08.164893 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.164796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5qtk\" (UniqueName: \"kubernetes.io/projected/961602ee-dd37-4363-b81b-1a7f24ca17cf-kube-api-access-r5qtk\") pod \"961602ee-dd37-4363-b81b-1a7f24ca17cf\" (UID: \"961602ee-dd37-4363-b81b-1a7f24ca17cf\") "
Jul 22 12:48:08.167019 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.166989    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/961602ee-dd37-4363-b81b-1a7f24ca17cf-kube-api-access-r5qtk" (OuterVolumeSpecName: "kube-api-access-r5qtk") pod "961602ee-dd37-4363-b81b-1a7f24ca17cf" (UID: "961602ee-dd37-4363-b81b-1a7f24ca17cf"). InnerVolumeSpecName "kube-api-access-r5qtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:48:08.167140 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.167042    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/961602ee-dd37-4363-b81b-1a7f24ca17cf-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "961602ee-dd37-4363-b81b-1a7f24ca17cf" (UID: "961602ee-dd37-4363-b81b-1a7f24ca17cf"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:48:08.265547 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.265521    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5qtk\" (UniqueName: \"kubernetes.io/projected/961602ee-dd37-4363-b81b-1a7f24ca17cf-kube-api-access-r5qtk\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:48:08.265547 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.265544    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/961602ee-dd37-4363-b81b-1a7f24ca17cf-webhook-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:48:08.537359 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.537327    2563 generic.go:356] "Generic (PLEG): container finished" podID="961602ee-dd37-4363-b81b-1a7f24ca17cf" containerID="ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3" exitCode=0
Jul 22 12:48:08.537843 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.537383    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz"
Jul 22 12:48:08.537843 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.537415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz" event={"ID":"961602ee-dd37-4363-b81b-1a7f24ca17cf","Type":"ContainerDied","Data":"ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3"}
Jul 22 12:48:08.537843 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.537463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fbb74bd86-5dbsz" event={"ID":"961602ee-dd37-4363-b81b-1a7f24ca17cf","Type":"ContainerDied","Data":"e3b93800dc8610b99eee2dc3ce30154d1e1f0ee2231f08aebe8bb88444dbf93c"}
Jul 22 12:48:08.537843 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.537486    2563 scope.go:117] "RemoveContainer" containerID="ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3"
Jul 22 12:48:08.546598 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.546580    2563 scope.go:117] "RemoveContainer" containerID="ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3"
Jul 22 12:48:08.546878 ip-10-0-128-197 kubenswrapper[2563]: E0722 12:48:08.546858    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3\": container with ID starting with ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3 not found: ID does not exist" containerID="ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3"
Jul 22 12:48:08.547008 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.546885    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3"} err="failed to get container status \"ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3\": rpc error: code = NotFound desc = could not find container \"ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3\": container with ID starting with ac15acb757f2282fe28892e668c2fa7a43bcfbc34c7011e04dbd4cc2178c1df3 not found: ID does not exist"
Jul 22 12:48:08.559001 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.558958    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fbb74bd86-5dbsz"]
Jul 22 12:48:08.560930 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:08.560911    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fbb74bd86-5dbsz"]
Jul 22 12:48:10.017478 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:10.017447    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="961602ee-dd37-4363-b81b-1a7f24ca17cf" path="/var/lib/kubelet/pods/961602ee-dd37-4363-b81b-1a7f24ca17cf/volumes"
Jul 22 12:48:27.415339 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.415296    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"]
Jul 22 12:48:27.415913 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.415795    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="961602ee-dd37-4363-b81b-1a7f24ca17cf" containerName="manager"
Jul 22 12:48:27.415913 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.415814    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="961602ee-dd37-4363-b81b-1a7f24ca17cf" containerName="manager"
Jul 22 12:48:27.415913 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.415904    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="961602ee-dd37-4363-b81b-1a7f24ca17cf" containerName="manager"
Jul 22 12:48:27.418733 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.418713    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.422128 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.422106    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Jul 22 12:48:27.422583 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.422565    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8dbk8\""
Jul 22 12:48:27.434798 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.434778    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"]
Jul 22 12:48:27.515395 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.515365    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10a2b508-9335-4069-ad7f-851dcd000389-maas-api-tls\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.515523 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.515433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10a2b508-9335-4069-ad7f-851dcd000389-openshift-service-ca\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.515523 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.515464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc2fv\" (UniqueName: \"kubernetes.io/projected/10a2b508-9335-4069-ad7f-851dcd000389-kube-api-access-pc2fv\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.615895 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.615858    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10a2b508-9335-4069-ad7f-851dcd000389-openshift-service-ca\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.615895 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.615887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc2fv\" (UniqueName: \"kubernetes.io/projected/10a2b508-9335-4069-ad7f-851dcd000389-kube-api-access-pc2fv\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.616153 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.615931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10a2b508-9335-4069-ad7f-851dcd000389-maas-api-tls\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.616538 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.616510    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/10a2b508-9335-4069-ad7f-851dcd000389-openshift-service-ca\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.618425 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.618406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/10a2b508-9335-4069-ad7f-851dcd000389-maas-api-tls\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.626675 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.626654    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc2fv\" (UniqueName: \"kubernetes.io/projected/10a2b508-9335-4069-ad7f-851dcd000389-kube-api-access-pc2fv\") pod \"maas-api-684f74c5d4-7qpm8\" (UID: \"10a2b508-9335-4069-ad7f-851dcd000389\") " pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.728126 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.728100    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:27.862084 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:27.861920    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"]
Jul 22 12:48:27.864332 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:48:27.864306    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10a2b508_9335_4069_ad7f_851dcd000389.slice/crio-3c75e801bc697b0c431fdbb8334bdb6d6d7050ab8cca684081b2a4a2e691f882 WatchSource:0}: Error finding container 3c75e801bc697b0c431fdbb8334bdb6d6d7050ab8cca684081b2a4a2e691f882: Status 404 returned error can't find the container with id 3c75e801bc697b0c431fdbb8334bdb6d6d7050ab8cca684081b2a4a2e691f882
Jul 22 12:48:28.043372 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.043305    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"]
Jul 22 12:48:28.048106 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.048089    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.052290 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.052273    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-856rf\""
Jul 22 12:48:28.052860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.052844    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 22 12:48:28.066697 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.066676    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"]
Jul 22 12:48:28.068824 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.068800    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-gxkfc"]
Jul 22 12:48:28.073996 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.073978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.084186 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.084168    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-gxkfc"]
Jul 22 12:48:28.119335 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.119310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t72hr\" (UniqueName: \"kubernetes.io/projected/c68873a9-6030-4711-9caf-b29cece27106-kube-api-access-t72hr\") pod \"payload-pre-processing-7677b774bb-pdkgm\" (UID: \"c68873a9-6030-4711-9caf-b29cece27106\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.119476 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.119360    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9mrj\" (UniqueName: \"kubernetes.io/projected/91c7f6e1-c5f9-4e88-9aed-3a529646e1c4-kube-api-access-x9mrj\") pod \"payload-processing-5bf47cd988-gxkfc\" (UID: \"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4\") " pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.119666 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.119634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68873a9-6030-4711-9caf-b29cece27106-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-pdkgm\" (UID: \"c68873a9-6030-4711-9caf-b29cece27106\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.119860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.119837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c7f6e1-c5f9-4e88-9aed-3a529646e1c4-plugins-config-volume\") pod \"payload-processing-5bf47cd988-gxkfc\" (UID: \"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4\") " pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.221145 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.221115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c7f6e1-c5f9-4e88-9aed-3a529646e1c4-plugins-config-volume\") pod \"payload-processing-5bf47cd988-gxkfc\" (UID: \"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4\") " pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.221284 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.221162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t72hr\" (UniqueName: \"kubernetes.io/projected/c68873a9-6030-4711-9caf-b29cece27106-kube-api-access-t72hr\") pod \"payload-pre-processing-7677b774bb-pdkgm\" (UID: \"c68873a9-6030-4711-9caf-b29cece27106\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.221284 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.221189    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9mrj\" (UniqueName: \"kubernetes.io/projected/91c7f6e1-c5f9-4e88-9aed-3a529646e1c4-kube-api-access-x9mrj\") pod \"payload-processing-5bf47cd988-gxkfc\" (UID: \"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4\") " pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.221284 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.221213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68873a9-6030-4711-9caf-b29cece27106-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-pdkgm\" (UID: \"c68873a9-6030-4711-9caf-b29cece27106\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.221921 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.221892    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/91c7f6e1-c5f9-4e88-9aed-3a529646e1c4-plugins-config-volume\") pod \"payload-processing-5bf47cd988-gxkfc\" (UID: \"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4\") " pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.221921 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.221910    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c68873a9-6030-4711-9caf-b29cece27106-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-pdkgm\" (UID: \"c68873a9-6030-4711-9caf-b29cece27106\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.230154 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.230133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9mrj\" (UniqueName: \"kubernetes.io/projected/91c7f6e1-c5f9-4e88-9aed-3a529646e1c4-kube-api-access-x9mrj\") pod \"payload-processing-5bf47cd988-gxkfc\" (UID: \"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4\") " pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.230312 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.230243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t72hr\" (UniqueName: \"kubernetes.io/projected/c68873a9-6030-4711-9caf-b29cece27106-kube-api-access-t72hr\") pod \"payload-pre-processing-7677b774bb-pdkgm\" (UID: \"c68873a9-6030-4711-9caf-b29cece27106\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.358990 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.358916    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:28.384430 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.384406    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:28.508932 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.508904    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"]
Jul 22 12:48:28.511220 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:48:28.511171    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc68873a9_6030_4711_9caf_b29cece27106.slice/crio-52522d44c70344c2824f83b0af3c7677b81259525f1e95240ce61018959814bb WatchSource:0}: Error finding container 52522d44c70344c2824f83b0af3c7677b81259525f1e95240ce61018959814bb: Status 404 returned error can't find the container with id 52522d44c70344c2824f83b0af3c7677b81259525f1e95240ce61018959814bb
Jul 22 12:48:28.530124 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.530101    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-gxkfc"]
Jul 22 12:48:28.532034 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:48:28.532009    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91c7f6e1_c5f9_4e88_9aed_3a529646e1c4.slice/crio-4d0dab44a82cbb22cc3eebc9cbeff553a8a7ef6da421816cf35c8f16124d2ca9 WatchSource:0}: Error finding container 4d0dab44a82cbb22cc3eebc9cbeff553a8a7ef6da421816cf35c8f16124d2ca9: Status 404 returned error can't find the container with id 4d0dab44a82cbb22cc3eebc9cbeff553a8a7ef6da421816cf35c8f16124d2ca9
Jul 22 12:48:28.624535 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.624425    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm" event={"ID":"c68873a9-6030-4711-9caf-b29cece27106","Type":"ContainerStarted","Data":"52522d44c70344c2824f83b0af3c7677b81259525f1e95240ce61018959814bb"}
Jul 22 12:48:28.625898 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.625873    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc" event={"ID":"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4","Type":"ContainerStarted","Data":"4d0dab44a82cbb22cc3eebc9cbeff553a8a7ef6da421816cf35c8f16124d2ca9"}
Jul 22 12:48:28.627400 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:28.627371    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8" event={"ID":"10a2b508-9335-4069-ad7f-851dcd000389","Type":"ContainerStarted","Data":"3c75e801bc697b0c431fdbb8334bdb6d6d7050ab8cca684081b2a4a2e691f882"}
Jul 22 12:48:30.641163 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:30.641124    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8" event={"ID":"10a2b508-9335-4069-ad7f-851dcd000389","Type":"ContainerStarted","Data":"ccb1207aa19b505b0ef34f6569ef235f33e78fca98c7b388fb8651714fbbfb9b"}
Jul 22 12:48:30.641633 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:30.641592    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:30.668670 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:30.666141    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8" podStartSLOduration=1.787286253 podStartE2EDuration="3.666120905s" totalImagesPullingTime="1.878834652s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:48:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:48:30.665128879 +0000 UTC m=+695.130026933" watchObservedRunningTime="2026-07-22 12:48:30.666120905 +0000 UTC m=+695.131018955"
Jul 22 12:48:32.654387 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:32.654347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc" event={"ID":"91c7f6e1-c5f9-4e88-9aed-3a529646e1c4","Type":"ContainerStarted","Data":"52b1706143848bc2b51129bb74604f4df9b04b1f623517e77ad199fdcca9263e"}
Jul 22 12:48:32.655207 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:32.655179    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:48:32.656960 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:32.656932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm" event={"ID":"c68873a9-6030-4711-9caf-b29cece27106","Type":"ContainerStarted","Data":"56078a93a2260388ee69b6da52115b787cba3c10f2f745ed07baef79e23ae748"}
Jul 22 12:48:32.657429 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:32.657403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:32.673263 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:32.673214    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc" podStartSLOduration=1.425560212 podStartE2EDuration="4.673196461s" totalImagesPullingTime="3.247636249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:48:32.671597311 +0000 UTC m=+697.136495358" watchObservedRunningTime="2026-07-22 12:48:32.673196461 +0000 UTC m=+697.138094511"
Jul 22 12:48:32.688964 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:32.688920    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm" podStartSLOduration=1.432428171 podStartE2EDuration="4.688908076s" totalImagesPullingTime="3.256479905s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:48:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:48:32.68800774 +0000 UTC m=+697.152905790" watchObservedRunningTime="2026-07-22 12:48:32.688908076 +0000 UTC m=+697.153806124"
Jul 22 12:48:36.652917 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:36.652885    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-684f74c5d4-7qpm8"
Jul 22 12:48:36.676671 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:36.676645    2563 generic.go:356] "Generic (PLEG): container finished" podID="d664d98d-27de-4f92-ac66-bc32e06d6209" containerID="25756890d13681a053f47b66979024f0828919030947331370464fea8c59d600" exitCode=0
Jul 22 12:48:36.676860 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:36.676696    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" event={"ID":"d664d98d-27de-4f92-ac66-bc32e06d6209","Type":"ContainerDied","Data":"25756890d13681a053f47b66979024f0828919030947331370464fea8c59d600"}
Jul 22 12:48:36.677187 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:36.677169    2563 scope.go:117] "RemoveContainer" containerID="25756890d13681a053f47b66979024f0828919030947331370464fea8c59d600"
Jul 22 12:48:36.678369 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:36.678340    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:48:36.678440 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:36.678419    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:48:36.678483 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:36.678465    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 12:48:37.683371 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:37.683340    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" event={"ID":"d664d98d-27de-4f92-ac66-bc32e06d6209","Type":"ContainerStarted","Data":"40d1dbc92d1087dbbd0c0a41f99eb966853b59da56274fa83b2feffb37012369"}
Jul 22 12:48:38.127900 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:38.127869    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:48:38.129249 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:38.129220    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4jkgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:48:38.129399 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:38.129278    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" podUID="d664d98d-27de-4f92-ac66-bc32e06d6209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:48:39.128204 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:39.128160    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4jkgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:48:39.128702 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:39.128224    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" podUID="d664d98d-27de-4f92-ac66-bc32e06d6209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:48:40.128483 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:40.128431    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4jkgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:48:40.128911 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:40.128498    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" podUID="d664d98d-27de-4f92-ac66-bc32e06d6209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:48:41.128318 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:41.128281    2563 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-4jkgx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused" start-of-body=
Jul 22 12:48:41.128499 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:41.128340    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx" podUID="d664d98d-27de-4f92-ac66-bc32e06d6209" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.31:15021/healthz/ready\": dial tcp 10.132.0.31:15021: connect: connection refused"
Jul 22 12:48:42.185485 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:42.185454    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:48:42.185943 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:42.185786    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:48:42.186816 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:42.186792    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-4jkgx"
Jul 22 12:48:44.668847 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:44.668817    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-pdkgm"
Jul 22 12:48:44.669235 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:48:44.668885    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-gxkfc"
Jul 22 12:49:31.016148 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.016119    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"]
Jul 22 12:49:31.019871 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.019841    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.022733 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.022712    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 22 12:49:31.022858 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.022735    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sg465\""
Jul 22 12:49:31.023639 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.023614    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 22 12:49:31.023797 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.023780    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 22 12:49:31.032856 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.032838    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"]
Jul 22 12:49:31.160638 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.160606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gscpn\" (UniqueName: \"kubernetes.io/projected/f7e87297-7204-43d7-9839-1171ee1092ce-kube-api-access-gscpn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.160979 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.160948    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.161120 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.161007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e87297-7204-43d7-9839-1171ee1092ce-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.161120 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.161051    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.161120 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.161094    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.161625 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.161600    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.262933 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.262895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.262933 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.262938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.263110 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.262990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.263110 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.263026    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gscpn\" (UniqueName: \"kubernetes.io/projected/f7e87297-7204-43d7-9839-1171ee1092ce-kube-api-access-gscpn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.263174 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.263155    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.263212 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.263194    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e87297-7204-43d7-9839-1171ee1092ce-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.263367 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.263345    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.263435 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.263414    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.263528 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.263508    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.265412 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.265382    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f7e87297-7204-43d7-9839-1171ee1092ce-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.265634 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.265618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f7e87297-7204-43d7-9839-1171ee1092ce-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.271389 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.271334    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gscpn\" (UniqueName: \"kubernetes.io/projected/f7e87297-7204-43d7-9839-1171ee1092ce-kube-api-access-gscpn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v5z9g\" (UID: \"f7e87297-7204-43d7-9839-1171ee1092ce\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.332396 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.332375    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:31.464394 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.464365    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"]
Jul 22 12:49:31.466279 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:49:31.466251    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7e87297_7204_43d7_9839_1171ee1092ce.slice/crio-0c9b14cfaafbb165a1a0039f98e64e95712fc008d8f6d37ed2d2d59318a5d011 WatchSource:0}: Error finding container 0c9b14cfaafbb165a1a0039f98e64e95712fc008d8f6d37ed2d2d59318a5d011: Status 404 returned error can't find the container with id 0c9b14cfaafbb165a1a0039f98e64e95712fc008d8f6d37ed2d2d59318a5d011
Jul 22 12:49:31.467614 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.467594    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 12:49:31.943669 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:31.943634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g" event={"ID":"f7e87297-7204-43d7-9839-1171ee1092ce","Type":"ContainerStarted","Data":"0c9b14cfaafbb165a1a0039f98e64e95712fc008d8f6d37ed2d2d59318a5d011"}
Jul 22 12:49:32.949760 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:32.949712    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g" event={"ID":"f7e87297-7204-43d7-9839-1171ee1092ce","Type":"ContainerStarted","Data":"70a7f0d46d2408b8642deab088ac687a5d18932b5c07d25885c00b4aed4892ba"}
Jul 22 12:49:32.971353 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:32.971305    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g" podStartSLOduration=1.655455115 podStartE2EDuration="2.971289381s" totalImagesPullingTime="1.315834266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:49:32.970823518 +0000 UTC m=+757.435721568" watchObservedRunningTime="2026-07-22 12:49:32.971289381 +0000 UTC m=+757.436187429"
Jul 22 12:49:41.333497 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:41.333453    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:41.334075 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:41.333553    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:41.346046 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:41.346027    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:42.000417 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:42.000384    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v5z9g"
Jul 22 12:49:49.229261 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.229232    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"]
Jul 22 12:49:49.236572 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.236551    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.241389 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.241370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 22 12:49:49.247491 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.247466    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"]
Jul 22 12:49:49.407914 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.407882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8bmq\" (UniqueName: \"kubernetes.io/projected/8a785aec-ec3a-4e10-b41b-8d31b77d6227-kube-api-access-f8bmq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.408086 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.407919    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a785aec-ec3a-4e10-b41b-8d31b77d6227-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.408086 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.407951    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.408086 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.408015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.408086 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.408068    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.408221 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.408103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.508944 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.508867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.508944 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.508931    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8bmq\" (UniqueName: \"kubernetes.io/projected/8a785aec-ec3a-4e10-b41b-8d31b77d6227-kube-api-access-f8bmq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.509139 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.508953    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a785aec-ec3a-4e10-b41b-8d31b77d6227-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.509139 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.508981    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.509139 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.508999    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.509139 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.509021    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.509659 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.509625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.509811 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.509662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.509918 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.509894    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.511348 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.511327    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a785aec-ec3a-4e10-b41b-8d31b77d6227-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.511606 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.511591    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a785aec-ec3a-4e10-b41b-8d31b77d6227-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.517646 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.517625    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8bmq\" (UniqueName: \"kubernetes.io/projected/8a785aec-ec3a-4e10-b41b-8d31b77d6227-kube-api-access-f8bmq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-2vm7k\" (UID: \"8a785aec-ec3a-4e10-b41b-8d31b77d6227\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.546316 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.546289    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:49.686998 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:49.686973    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"]
Jul 22 12:49:49.688738 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:49:49.688713    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a785aec_ec3a_4e10_b41b_8d31b77d6227.slice/crio-a2694e09301b802f9e8bccc08922e83507a4860fc13621cc918374fb0bb91d4f WatchSource:0}: Error finding container a2694e09301b802f9e8bccc08922e83507a4860fc13621cc918374fb0bb91d4f: Status 404 returned error can't find the container with id a2694e09301b802f9e8bccc08922e83507a4860fc13621cc918374fb0bb91d4f
Jul 22 12:49:50.028956 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:50.028932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k" event={"ID":"8a785aec-ec3a-4e10-b41b-8d31b77d6227","Type":"ContainerStarted","Data":"85b36c20646ef1ad80852f9ee85fab4a1e4dd368c79e9377b98a227de1966739"}
Jul 22 12:49:50.029050 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:50.028976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k" event={"ID":"8a785aec-ec3a-4e10-b41b-8d31b77d6227","Type":"ContainerStarted","Data":"a2694e09301b802f9e8bccc08922e83507a4860fc13621cc918374fb0bb91d4f"}
Jul 22 12:49:50.047542 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:50.047497    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k" podStartSLOduration=0.770732122 podStartE2EDuration="1.047483831s" totalImagesPullingTime="276.751709ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:49:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:49:50.046577125 +0000 UTC m=+774.511475175" watchObservedRunningTime="2026-07-22 12:49:50.047483831 +0000 UTC m=+774.512381879"
Jul 22 12:49:59.547433 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:59.547401    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:59.547845 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:59.547480    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:49:59.559784 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:49:59.559741    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:50:00.083662 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:00.083634    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-2vm7k"
Jul 22 12:50:04.504308 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.504275    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"]
Jul 22 12:50:04.508250 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.508230    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.510900 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.510880    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 22 12:50:04.522846 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.522825    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"]
Jul 22 12:50:04.525125 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.525095    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.525236 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.525158    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.525236 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.525201    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.525310 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.525261    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.525310 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.525287    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mtrkk\" (UniqueName: \"kubernetes.io/projected/238c37cf-948a-4c13-8880-6cd2f0eb1c15-kube-api-access-mtrkk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.525373 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.525330    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/238c37cf-948a-4c13-8880-6cd2f0eb1c15-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.625830 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.625800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.625989 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.625841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.625989 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.625867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.625989 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.625887    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mtrkk\" (UniqueName: \"kubernetes.io/projected/238c37cf-948a-4c13-8880-6cd2f0eb1c15-kube-api-access-mtrkk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.625989 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.625928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/238c37cf-948a-4c13-8880-6cd2f0eb1c15-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.626184 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.625996    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.626260 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.626240    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.626319 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.626290    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.626375 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.626346    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.628338 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.628316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/238c37cf-948a-4c13-8880-6cd2f0eb1c15-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.628503 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.628484    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/238c37cf-948a-4c13-8880-6cd2f0eb1c15-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.635632 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.635610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mtrkk\" (UniqueName: \"kubernetes.io/projected/238c37cf-948a-4c13-8880-6cd2f0eb1c15-kube-api-access-mtrkk\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8\" (UID: \"238c37cf-948a-4c13-8880-6cd2f0eb1c15\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.821351 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.821262    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:04.956976 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:04.956952    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"]
Jul 22 12:50:04.958919 ip-10-0-128-197 kubenswrapper[2563]: W0722 12:50:04.958895    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod238c37cf_948a_4c13_8880_6cd2f0eb1c15.slice/crio-4b385fb09cb8288c2fd93ba1b314294a396e8f74a06c9c484fa141ee031352df WatchSource:0}: Error finding container 4b385fb09cb8288c2fd93ba1b314294a396e8f74a06c9c484fa141ee031352df: Status 404 returned error can't find the container with id 4b385fb09cb8288c2fd93ba1b314294a396e8f74a06c9c484fa141ee031352df
Jul 22 12:50:05.094563 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:05.094486    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8" event={"ID":"238c37cf-948a-4c13-8880-6cd2f0eb1c15","Type":"ContainerStarted","Data":"4b385fb09cb8288c2fd93ba1b314294a396e8f74a06c9c484fa141ee031352df"}
Jul 22 12:50:06.100641 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:06.100602    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8" event={"ID":"238c37cf-948a-4c13-8880-6cd2f0eb1c15","Type":"ContainerStarted","Data":"d34c532cb9081bf86f515b224f558b537977e5334487dd5d100603a23411d0e1"}
Jul 22 12:50:06.119341 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:06.119294    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8" podStartSLOduration=1.81433985 podStartE2EDuration="2.119278982s" totalImagesPullingTime="304.939132ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 12:50:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 12:50:06.118437766 +0000 UTC m=+790.583335816" watchObservedRunningTime="2026-07-22 12:50:06.119278982 +0000 UTC m=+790.584177030"
Jul 22 12:50:14.822150 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:14.822117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:14.822574 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:14.822274    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:14.835030 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:14.835011    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:15.152655 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:15.152578    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8"
Jul 22 12:50:39.049420 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.049383    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8667994f5d-gfmkr"]
Jul 22 12:50:39.050040 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.049612    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-8667994f5d-gfmkr" podUID="c8258fe8-c06a-44fd-b801-31fe2ab15c4f" containerName="authorino" containerID="cri-o://7b63598c1abbb9225c2647f98e03e3088e38bc9d6e59583cb540e3bef954182c" gracePeriod=30
Jul 22 12:50:39.252100 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.252070    2563 generic.go:356] "Generic (PLEG): container finished" podID="c8258fe8-c06a-44fd-b801-31fe2ab15c4f" containerID="7b63598c1abbb9225c2647f98e03e3088e38bc9d6e59583cb540e3bef954182c" exitCode=0
Jul 22 12:50:39.252256 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.252170    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8667994f5d-gfmkr" event={"ID":"c8258fe8-c06a-44fd-b801-31fe2ab15c4f","Type":"ContainerDied","Data":"7b63598c1abbb9225c2647f98e03e3088e38bc9d6e59583cb540e3bef954182c"}
Jul 22 12:50:39.302817 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.302761    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:50:39.404305 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.404274    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-tls-cert\") pod \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") "
Jul 22 12:50:39.404483 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.404407    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjwbt\" (UniqueName: \"kubernetes.io/projected/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-kube-api-access-pjwbt\") pod \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\" (UID: \"c8258fe8-c06a-44fd-b801-31fe2ab15c4f\") "
Jul 22 12:50:39.406472 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.406439    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-kube-api-access-pjwbt" (OuterVolumeSpecName: "kube-api-access-pjwbt") pod "c8258fe8-c06a-44fd-b801-31fe2ab15c4f" (UID: "c8258fe8-c06a-44fd-b801-31fe2ab15c4f"). InnerVolumeSpecName "kube-api-access-pjwbt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 12:50:39.415130 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.415103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "c8258fe8-c06a-44fd-b801-31fe2ab15c4f" (UID: "c8258fe8-c06a-44fd-b801-31fe2ab15c4f"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 12:50:39.505201 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.505172    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjwbt\" (UniqueName: \"kubernetes.io/projected/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-kube-api-access-pjwbt\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:50:39.505201 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:39.505198    2563 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c8258fe8-c06a-44fd-b801-31fe2ab15c4f-tls-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 12:50:40.257806 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:40.257776    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8667994f5d-gfmkr"
Jul 22 12:50:40.258263 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:40.257804    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8667994f5d-gfmkr" event={"ID":"c8258fe8-c06a-44fd-b801-31fe2ab15c4f","Type":"ContainerDied","Data":"1b3297dfbfe89adb47680bdc0b1f4646c1dce8205bc6a4172195458b52d81a84"}
Jul 22 12:50:40.258263 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:40.257858    2563 scope.go:117] "RemoveContainer" containerID="7b63598c1abbb9225c2647f98e03e3088e38bc9d6e59583cb540e3bef954182c"
Jul 22 12:50:40.277939 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:40.277891    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8667994f5d-gfmkr"]
Jul 22 12:50:40.285829 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:40.285806    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8667994f5d-gfmkr"]
Jul 22 12:50:42.018470 ip-10-0-128-197 kubenswrapper[2563]: I0722 12:50:42.018434    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8258fe8-c06a-44fd-b801-31fe2ab15c4f" path="/var/lib/kubelet/pods/c8258fe8-c06a-44fd-b801-31fe2ab15c4f/volumes"
Jul 22 13:01:22.694009 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.693924    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"]
Jul 22 13:01:22.694573 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.694461    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8258fe8-c06a-44fd-b801-31fe2ab15c4f" containerName="authorino"
Jul 22 13:01:22.694573 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.694480    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8258fe8-c06a-44fd-b801-31fe2ab15c4f" containerName="authorino"
Jul 22 13:01:22.694688 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.694581    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8258fe8-c06a-44fd-b801-31fe2ab15c4f" containerName="authorino"
Jul 22 13:01:22.697731 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.697712    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.700297 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.700272    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-eb01b094-kserve-self-signed-certs\""
Jul 22 13:01:22.707636 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.707616    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"]
Jul 22 13:01:22.802403 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.802371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bb8t\" (UniqueName: \"kubernetes.io/projected/dd82b07f-35e6-40b6-94ec-021cde71cf6a-kube-api-access-6bb8t\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.802549 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.802408    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-home\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.802549 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.802430    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tmp-dir\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.802549 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.802479    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-dshm\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.802549 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.802522    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tls-certs\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.802681 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.802572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-model-cache\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903436 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903401    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bb8t\" (UniqueName: \"kubernetes.io/projected/dd82b07f-35e6-40b6-94ec-021cde71cf6a-kube-api-access-6bb8t\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903608 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-home\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903608 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tmp-dir\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903608 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-dshm\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903608 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903545    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tls-certs\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903608 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903605    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-model-cache\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903914 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-home\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.903995 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.903971    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tmp-dir\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.904073 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.904058    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-model-cache\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.905818 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.905788    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-dshm\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.906294 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.906272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tls-certs\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:22.912450 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:22.912430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bb8t\" (UniqueName: \"kubernetes.io/projected/dd82b07f-35e6-40b6-94ec-021cde71cf6a-kube-api-access-6bb8t\") pod \"e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") " pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:23.007696 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:23.007607    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:23.142476 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:23.142452    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"]
Jul 22 13:01:23.143825 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:01:23.143798    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddd82b07f_35e6_40b6_94ec_021cde71cf6a.slice/crio-de56483cbd6aa1871bfc3a2d46d0aff803e8ba2b62d410b34e60bf8c5e608d20 WatchSource:0}: Error finding container de56483cbd6aa1871bfc3a2d46d0aff803e8ba2b62d410b34e60bf8c5e608d20: Status 404 returned error can't find the container with id de56483cbd6aa1871bfc3a2d46d0aff803e8ba2b62d410b34e60bf8c5e608d20
Jul 22 13:01:23.177446 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:23.177421    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz" event={"ID":"dd82b07f-35e6-40b6-94ec-021cde71cf6a","Type":"ContainerStarted","Data":"de56483cbd6aa1871bfc3a2d46d0aff803e8ba2b62d410b34e60bf8c5e608d20"}
Jul 22 13:01:24.186492 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:24.186458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz" event={"ID":"dd82b07f-35e6-40b6-94ec-021cde71cf6a","Type":"ContainerStarted","Data":"a0d07ef16ca814fbb85d4ac4de89809d746d7c8c7b5122bf6d31aa7ec44d4b5a"}
Jul 22 13:01:24.204709 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:24.204660    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz" podStartSLOduration=2.204644831 podStartE2EDuration="2.204644831s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:01:24.203546577 +0000 UTC m=+1468.668444639" watchObservedRunningTime="2026-07-22 13:01:24.204644831 +0000 UTC m=+1468.669542880"
Jul 22 13:01:33.007766 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:33.007707    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:33.008246 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:33.007781    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:33.020259 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:33.020231    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:33.239416 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:33.239383    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:01:45.987557 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:45.987521    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"]
Jul 22 13:01:45.991609 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:45.991590    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:45.994665 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:45.994647    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-eb01b094-kserve-self-signed-certs\""
Jul 22 13:01:46.010177 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.010151    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"]
Jul 22 13:01:46.102073 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.102029    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-model-cache\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.102226 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.102086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-home\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.102226 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.102156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-tmp-dir\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.102226 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.102197    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf4nr\" (UniqueName: \"kubernetes.io/projected/d5d6c036-516f-4e68-b498-ec4814a815b9-kube-api-access-cf4nr\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.102226 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.102219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-dshm\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.102390 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.102234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d6c036-516f-4e68-b498-ec4814a815b9-tls-certs\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.202893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.202850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-dshm\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.202893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.202896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d6c036-516f-4e68-b498-ec4814a815b9-tls-certs\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.203182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.202948    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-model-cache\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.203182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.202973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-home\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.203182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.203005    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-tmp-dir\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.203182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.203040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf4nr\" (UniqueName: \"kubernetes.io/projected/d5d6c036-516f-4e68-b498-ec4814a815b9-kube-api-access-cf4nr\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.203395 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.203342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-model-cache\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.203453 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.203391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-tmp-dir\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.203453 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.203436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-home\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.205252 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.205230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-dshm\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.205605 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.205585    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d6c036-516f-4e68-b498-ec4814a815b9-tls-certs\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.211296 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.211272    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf4nr\" (UniqueName: \"kubernetes.io/projected/d5d6c036-516f-4e68-b498-ec4814a815b9-kube-api-access-cf4nr\") pod \"e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") " pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.310088 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.310005    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:46.447402 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:46.447377    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"]
Jul 22 13:01:46.449973 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:01:46.449946    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5d6c036_516f_4e68_b498_ec4814a815b9.slice/crio-8de817a850e67f25eeb9f150585128c5a1cca9b63d1fb5cd2615620194826061 WatchSource:0}: Error finding container 8de817a850e67f25eeb9f150585128c5a1cca9b63d1fb5cd2615620194826061: Status 404 returned error can't find the container with id 8de817a850e67f25eeb9f150585128c5a1cca9b63d1fb5cd2615620194826061
Jul 22 13:01:47.291840 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:47.291802    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f" event={"ID":"d5d6c036-516f-4e68-b498-ec4814a815b9","Type":"ContainerStarted","Data":"3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8"}
Jul 22 13:01:47.291840 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:47.291848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f" event={"ID":"d5d6c036-516f-4e68-b498-ec4814a815b9","Type":"ContainerStarted","Data":"8de817a850e67f25eeb9f150585128c5a1cca9b63d1fb5cd2615620194826061"}
Jul 22 13:01:47.315039 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:47.314991    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f" podStartSLOduration=2.314977016 podStartE2EDuration="2.314977016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:01:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:01:47.312638306 +0000 UTC m=+1491.777536354" watchObservedRunningTime="2026-07-22 13:01:47.314977016 +0000 UTC m=+1491.779875135"
Jul 22 13:01:56.311152 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:56.311117    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:56.311804 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:56.311177    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:56.325823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:56.325802    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:01:56.345387 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:01:56.345365    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:02:13.307296 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:13.307261    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"]
Jul 22 13:02:14.625510 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:14.625472    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"]
Jul 22 13:02:28.328773 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.328710    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz" podUID="dd82b07f-35e6-40b6-94ec-021cde71cf6a" containerName="main" containerID="cri-o://a0d07ef16ca814fbb85d4ac4de89809d746d7c8c7b5122bf6d31aa7ec44d4b5a" gracePeriod=45
Jul 22 13:02:28.484807 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.484774    2563 generic.go:356] "Generic (PLEG): container finished" podID="dd82b07f-35e6-40b6-94ec-021cde71cf6a" containerID="a0d07ef16ca814fbb85d4ac4de89809d746d7c8c7b5122bf6d31aa7ec44d4b5a" exitCode=0
Jul 22 13:02:28.484966 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.484839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz" event={"ID":"dd82b07f-35e6-40b6-94ec-021cde71cf6a","Type":"ContainerDied","Data":"a0d07ef16ca814fbb85d4ac4de89809d746d7c8c7b5122bf6d31aa7ec44d4b5a"}
Jul 22 13:02:28.583544 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.583491    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:02:28.680702 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.680672    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-dshm\") pod \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") "
Jul 22 13:02:28.680879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.680711    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-home\") pod \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") "
Jul 22 13:02:28.680879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.680736    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tmp-dir\") pod \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") "
Jul 22 13:02:28.680879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.680787    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-model-cache\") pod \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") "
Jul 22 13:02:28.680879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.680831    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tls-certs\") pod \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") "
Jul 22 13:02:28.680879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.680859    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6bb8t\" (UniqueName: \"kubernetes.io/projected/dd82b07f-35e6-40b6-94ec-021cde71cf6a-kube-api-access-6bb8t\") pod \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\" (UID: \"dd82b07f-35e6-40b6-94ec-021cde71cf6a\") "
Jul 22 13:02:28.681185 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.680982    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-home" (OuterVolumeSpecName: "home") pod "dd82b07f-35e6-40b6-94ec-021cde71cf6a" (UID: "dd82b07f-35e6-40b6-94ec-021cde71cf6a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:28.681185 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.681030    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "dd82b07f-35e6-40b6-94ec-021cde71cf6a" (UID: "dd82b07f-35e6-40b6-94ec-021cde71cf6a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:28.681185 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.681046    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-model-cache" (OuterVolumeSpecName: "model-cache") pod "dd82b07f-35e6-40b6-94ec-021cde71cf6a" (UID: "dd82b07f-35e6-40b6-94ec-021cde71cf6a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:28.681185 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.681147    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:28.681185 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.681159    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:28.681185 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.681185    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:28.683062 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.683031    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "dd82b07f-35e6-40b6-94ec-021cde71cf6a" (UID: "dd82b07f-35e6-40b6-94ec-021cde71cf6a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:02:28.683062 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.683054    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-dshm" (OuterVolumeSpecName: "dshm") pod "dd82b07f-35e6-40b6-94ec-021cde71cf6a" (UID: "dd82b07f-35e6-40b6-94ec-021cde71cf6a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:28.683221 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.683085    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd82b07f-35e6-40b6-94ec-021cde71cf6a-kube-api-access-6bb8t" (OuterVolumeSpecName: "kube-api-access-6bb8t") pod "dd82b07f-35e6-40b6-94ec-021cde71cf6a" (UID: "dd82b07f-35e6-40b6-94ec-021cde71cf6a"). InnerVolumeSpecName "kube-api-access-6bb8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:02:28.782092 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.782068    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dd82b07f-35e6-40b6-94ec-021cde71cf6a-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:28.782092 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.782089    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dd82b07f-35e6-40b6-94ec-021cde71cf6a-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:28.782241 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:28.782100    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6bb8t\" (UniqueName: \"kubernetes.io/projected/dd82b07f-35e6-40b6-94ec-021cde71cf6a-kube-api-access-6bb8t\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:29.491098 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.491067    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz" event={"ID":"dd82b07f-35e6-40b6-94ec-021cde71cf6a","Type":"ContainerDied","Data":"de56483cbd6aa1871bfc3a2d46d0aff803e8ba2b62d410b34e60bf8c5e608d20"}
Jul 22 13:02:29.491506 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.491107    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"
Jul 22 13:02:29.491506 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.491116    2563 scope.go:117] "RemoveContainer" containerID="a0d07ef16ca814fbb85d4ac4de89809d746d7c8c7b5122bf6d31aa7ec44d4b5a"
Jul 22 13:02:29.512574 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.512545    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"]
Jul 22 13:02:29.519223 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.519199    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-eb01b094-kserve-d59dbdf99-4hcqz"]
Jul 22 13:02:29.646207 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.646162    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f" podUID="d5d6c036-516f-4e68-b498-ec4814a815b9" containerName="main" containerID="cri-o://3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8" gracePeriod=45
Jul 22 13:02:29.906436 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.906413    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:02:29.993365 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d6c036-516f-4e68-b498-ec4814a815b9-tls-certs\") pod \"d5d6c036-516f-4e68-b498-ec4814a815b9\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") "
Jul 22 13:02:29.993532 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993374    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-dshm\") pod \"d5d6c036-516f-4e68-b498-ec4814a815b9\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") "
Jul 22 13:02:29.993532 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993402    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-model-cache\") pod \"d5d6c036-516f-4e68-b498-ec4814a815b9\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") "
Jul 22 13:02:29.993532 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993454    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-home\") pod \"d5d6c036-516f-4e68-b498-ec4814a815b9\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") "
Jul 22 13:02:29.993532 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993473    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-tmp-dir\") pod \"d5d6c036-516f-4e68-b498-ec4814a815b9\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") "
Jul 22 13:02:29.993764 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993568    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cf4nr\" (UniqueName: \"kubernetes.io/projected/d5d6c036-516f-4e68-b498-ec4814a815b9-kube-api-access-cf4nr\") pod \"d5d6c036-516f-4e68-b498-ec4814a815b9\" (UID: \"d5d6c036-516f-4e68-b498-ec4814a815b9\") "
Jul 22 13:02:29.993764 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993709    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-home" (OuterVolumeSpecName: "home") pod "d5d6c036-516f-4e68-b498-ec4814a815b9" (UID: "d5d6c036-516f-4e68-b498-ec4814a815b9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:29.993877 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993736    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5d6c036-516f-4e68-b498-ec4814a815b9" (UID: "d5d6c036-516f-4e68-b498-ec4814a815b9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:29.993877 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993805    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5d6c036-516f-4e68-b498-ec4814a815b9" (UID: "d5d6c036-516f-4e68-b498-ec4814a815b9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:29.993987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993883    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:29.993987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993900    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:29.993987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.993912    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:29.995470 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.995451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-dshm" (OuterVolumeSpecName: "dshm") pod "d5d6c036-516f-4e68-b498-ec4814a815b9" (UID: "d5d6c036-516f-4e68-b498-ec4814a815b9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:02:29.995654 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.995631    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5d6c036-516f-4e68-b498-ec4814a815b9-kube-api-access-cf4nr" (OuterVolumeSpecName: "kube-api-access-cf4nr") pod "d5d6c036-516f-4e68-b498-ec4814a815b9" (UID: "d5d6c036-516f-4e68-b498-ec4814a815b9"). InnerVolumeSpecName "kube-api-access-cf4nr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:02:29.995714 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:29.995676    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5d6c036-516f-4e68-b498-ec4814a815b9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5d6c036-516f-4e68-b498-ec4814a815b9" (UID: "d5d6c036-516f-4e68-b498-ec4814a815b9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:02:30.018195 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.018135    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd82b07f-35e6-40b6-94ec-021cde71cf6a" path="/var/lib/kubelet/pods/dd82b07f-35e6-40b6-94ec-021cde71cf6a/volumes"
Jul 22 13:02:30.094531 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.094496    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cf4nr\" (UniqueName: \"kubernetes.io/projected/d5d6c036-516f-4e68-b498-ec4814a815b9-kube-api-access-cf4nr\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:30.094531 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.094531    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5d6c036-516f-4e68-b498-ec4814a815b9-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:30.094531 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.094545    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5d6c036-516f-4e68-b498-ec4814a815b9-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:02:30.496583 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.496550    2563 generic.go:356] "Generic (PLEG): container finished" podID="d5d6c036-516f-4e68-b498-ec4814a815b9" containerID="3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8" exitCode=0
Jul 22 13:02:30.497059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.496600    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"
Jul 22 13:02:30.497059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.496632    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f" event={"ID":"d5d6c036-516f-4e68-b498-ec4814a815b9","Type":"ContainerDied","Data":"3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8"}
Jul 22 13:02:30.497059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.496674    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f" event={"ID":"d5d6c036-516f-4e68-b498-ec4814a815b9","Type":"ContainerDied","Data":"8de817a850e67f25eeb9f150585128c5a1cca9b63d1fb5cd2615620194826061"}
Jul 22 13:02:30.497059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.496691    2563 scope.go:117] "RemoveContainer" containerID="3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8"
Jul 22 13:02:30.505067 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.505051    2563 scope.go:117] "RemoveContainer" containerID="3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8"
Jul 22 13:02:30.505324 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:02:30.505306    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8\": container with ID starting with 3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8 not found: ID does not exist" containerID="3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8"
Jul 22 13:02:30.505359 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.505333    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8"} err="failed to get container status \"3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8\": rpc error: code = NotFound desc = could not find container \"3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8\": container with ID starting with 3880cf95fc226b4b3c8c233334e02a27406ccdf2cc06973164d4640c71e71cc8 not found: ID does not exist"
Jul 22 13:02:30.514252 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.514232    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"]
Jul 22 13:02:30.518129 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:30.518109    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-eb01b094-kserve-9674cbc95-sx48f"]
Jul 22 13:02:32.018112 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:02:32.018079    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5d6c036-516f-4e68-b498-ec4814a815b9" path="/var/lib/kubelet/pods/d5d6c036-516f-4e68-b498-ec4814a815b9/volumes"
Jul 22 13:05:55.036494 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.036462    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"]
Jul 22 13:05:55.038879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.036853    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd82b07f-35e6-40b6-94ec-021cde71cf6a" containerName="main"
Jul 22 13:05:55.038879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.036866    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd82b07f-35e6-40b6-94ec-021cde71cf6a" containerName="main"
Jul 22 13:05:55.038879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.036882    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5d6c036-516f-4e68-b498-ec4814a815b9" containerName="main"
Jul 22 13:05:55.038879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.036887    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5d6c036-516f-4e68-b498-ec4814a815b9" containerName="main"
Jul 22 13:05:55.038879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.036953    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd82b07f-35e6-40b6-94ec-021cde71cf6a" containerName="main"
Jul 22 13:05:55.038879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.036962    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5d6c036-516f-4e68-b498-ec4814a815b9" containerName="main"
Jul 22 13:05:55.039869 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.039854    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.042498 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.042476    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7638e0e4-openshift-default-dockercfg-fnd8g\""
Jul 22 13:05:55.062823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.062798    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"]
Jul 22 13:05:55.198820 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.198786    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-podinfo\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.198980 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.198837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2gnd\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-kube-api-access-b2gnd\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.198980 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.198912    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-certs\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.198980 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.198941    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-data\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.198980 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.198973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istiod-ca-cert\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.199139 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.199005    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-socket\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.199139 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.199035    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-token\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.199139 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.199069    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-envoy\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.199139 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.199089    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-credential-socket\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300633 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istiod-ca-cert\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300633 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300590    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-socket\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300633 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300626    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-token\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300657    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-envoy\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300682    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-credential-socket\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-podinfo\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300810    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2gnd\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-kube-api-access-b2gnd\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.300895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300856    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-certs\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.301124 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.300893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-data\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.301251 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.301137    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-credential-socket\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.301251 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.301195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-data\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.301372 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.301306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-socket\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.301372 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.301329    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-certs\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.301479 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.301371    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istiod-ca-cert\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.303201 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.303179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-envoy\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.303552 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.303534    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-podinfo\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.309434 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.309406    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-token\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.309682 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.309666    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2gnd\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-kube-api-access-b2gnd\") pod \"e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") " pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.349895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.349871    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:55.497109 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.497085    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"]
Jul 22 13:05:55.499092 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:05:55.499041    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb71f851_1a96_4a82_b6c5_fa5a4d96ef87.slice/crio-a3b7575f15e5b73816b67c87d9a61d826107af12a1b3ba17466a857b7b42a224 WatchSource:0}: Error finding container a3b7575f15e5b73816b67c87d9a61d826107af12a1b3ba17466a857b7b42a224: Status 404 returned error can't find the container with id a3b7575f15e5b73816b67c87d9a61d826107af12a1b3ba17466a857b7b42a224
Jul 22 13:05:55.501285 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.501258    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:05:55.501384 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.501330    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:05:55.501384 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:55.501374    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:05:56.454947 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:56.454909    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" event={"ID":"db71f851-1a96-4a82-b6c5-fa5a4d96ef87","Type":"ContainerStarted","Data":"fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8"}
Jul 22 13:05:56.455409 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:56.454971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" event={"ID":"db71f851-1a96-4a82-b6c5-fa5a4d96ef87","Type":"ContainerStarted","Data":"a3b7575f15e5b73816b67c87d9a61d826107af12a1b3ba17466a857b7b42a224"}
Jul 22 13:05:56.477906 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:56.477852    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" podStartSLOduration=1.477836632 podStartE2EDuration="1.477836632s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:05:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:05:56.476128372 +0000 UTC m=+1740.941026434" watchObservedRunningTime="2026-07-22 13:05:56.477836632 +0000 UTC m=+1740.942734680"
Jul 22 13:05:57.350802 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:57.350741    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:57.355443 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:57.355414    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:57.459801 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:57.459764    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:57.460799 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:57.460776    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:05:58.033590 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.033559    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"]
Jul 22 13:05:58.037060 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.037037    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.040058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.040037    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7638e0e4-serving-cert\""
Jul 22 13:05:58.046484 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.046461    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"]
Jul 22 13:05:58.228504 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.228472    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzzfp\" (UniqueName: \"kubernetes.io/projected/449184db-c80d-4286-8b96-5f44a73c049c-kube-api-access-nzzfp\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.228699 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.228547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/449184db-c80d-4286-8b96-5f44a73c049c-maas-api-tls\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.228699 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.228570    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/449184db-c80d-4286-8b96-5f44a73c049c-openshift-service-ca\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.330138 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.330041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzzfp\" (UniqueName: \"kubernetes.io/projected/449184db-c80d-4286-8b96-5f44a73c049c-kube-api-access-nzzfp\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.330347 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.330140    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/449184db-c80d-4286-8b96-5f44a73c049c-maas-api-tls\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.330347 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.330169    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/449184db-c80d-4286-8b96-5f44a73c049c-openshift-service-ca\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.330983 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.330962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/449184db-c80d-4286-8b96-5f44a73c049c-openshift-service-ca\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.332853 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.332833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/449184db-c80d-4286-8b96-5f44a73c049c-maas-api-tls\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.339532 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.339509    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzzfp\" (UniqueName: \"kubernetes.io/projected/449184db-c80d-4286-8b96-5f44a73c049c-kube-api-access-nzzfp\") pod \"maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.348323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.348293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:58.698761 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.698716    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"]
Jul 22 13:05:58.700235 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:05:58.700204    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod449184db_c80d_4286_8b96_5f44a73c049c.slice/crio-e775c01b1772ad425d8377609610fb864c3eaa3af3bebd718c23d5a880ba4a99 WatchSource:0}: Error finding container e775c01b1772ad425d8377609610fb864c3eaa3af3bebd718c23d5a880ba4a99: Status 404 returned error can't find the container with id e775c01b1772ad425d8377609610fb864c3eaa3af3bebd718c23d5a880ba4a99
Jul 22 13:05:58.701803 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:58.701785    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 22 13:05:59.470158 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:59.470053    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp" event={"ID":"449184db-c80d-4286-8b96-5f44a73c049c","Type":"ContainerStarted","Data":"cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05"}
Jul 22 13:05:59.470158 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:59.470107    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp" event={"ID":"449184db-c80d-4286-8b96-5f44a73c049c","Type":"ContainerStarted","Data":"e775c01b1772ad425d8377609610fb864c3eaa3af3bebd718c23d5a880ba4a99"}
Jul 22 13:05:59.470370 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:59.470261    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:05:59.490539 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:05:59.490495    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp" podStartSLOduration=0.996286341 podStartE2EDuration="1.490478564s" totalImagesPullingTime="494.192223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:05:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:05:59.489477384 +0000 UTC m=+1743.954375432" watchObservedRunningTime="2026-07-22 13:05:59.490478564 +0000 UTC m=+1743.955376613"
Jul 22 13:06:05.480451 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:05.480424    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:06:12.461856 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.461819    2563 patch_prober.go:28] interesting pod/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 13:06:12.462262 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.461879    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 13:06:12.532843 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.532808    2563 generic.go:356] "Generic (PLEG): container finished" podID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerID="fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8" exitCode=0
Jul 22 13:06:12.532843 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.532843    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" event={"ID":"db71f851-1a96-4a82-b6c5-fa5a4d96ef87","Type":"ContainerDied","Data":"fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8"}
Jul 22 13:06:12.533199 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.533186    2563 scope.go:117] "RemoveContainer" containerID="fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8"
Jul 22 13:06:12.534242 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.534209    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:12.534314 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.534297    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:12.534355 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:12.534343    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:13.545417 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:13.545376    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" event={"ID":"db71f851-1a96-4a82-b6c5-fa5a4d96ef87","Type":"ContainerStarted","Data":"76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33"}
Jul 22 13:06:14.351268 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:14.351225    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:06:14.351490 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:14.351355    2563 patch_prober.go:28] interesting pod/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 13:06:14.351490 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:14.351401    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 13:06:15.350508 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:15.350472    2563 patch_prober.go:28] interesting pod/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 13:06:15.351132 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:15.351101    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 13:06:16.351584 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:16.351144    2563 patch_prober.go:28] interesting pod/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 13:06:16.351584 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:16.351206    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 13:06:17.350468 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:17.350419    2563 patch_prober.go:28] interesting pod/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Jul 22 13:06:17.350692 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:17.350492    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Jul 22 13:06:18.354844 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:18.354808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:06:18.355343 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:18.355324    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:06:18.356028 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:18.356009    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:06:33.849917 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:33.849883    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"]
Jul 22 13:06:33.850730 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:33.850694    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp" podUID="449184db-c80d-4286-8b96-5f44a73c049c" containerName="maas-api" containerID="cri-o://cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05" gracePeriod=30
Jul 22 13:06:34.107325 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.107269    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:06:34.236284 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.236252    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/449184db-c80d-4286-8b96-5f44a73c049c-maas-api-tls\") pod \"449184db-c80d-4286-8b96-5f44a73c049c\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") "
Jul 22 13:06:34.236455 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.236320    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/449184db-c80d-4286-8b96-5f44a73c049c-openshift-service-ca\") pod \"449184db-c80d-4286-8b96-5f44a73c049c\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") "
Jul 22 13:06:34.236455 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.236411    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nzzfp\" (UniqueName: \"kubernetes.io/projected/449184db-c80d-4286-8b96-5f44a73c049c-kube-api-access-nzzfp\") pod \"449184db-c80d-4286-8b96-5f44a73c049c\" (UID: \"449184db-c80d-4286-8b96-5f44a73c049c\") "
Jul 22 13:06:34.236657 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.236632    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/449184db-c80d-4286-8b96-5f44a73c049c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "449184db-c80d-4286-8b96-5f44a73c049c" (UID: "449184db-c80d-4286-8b96-5f44a73c049c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:06:34.238526 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.238505    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/449184db-c80d-4286-8b96-5f44a73c049c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "449184db-c80d-4286-8b96-5f44a73c049c" (UID: "449184db-c80d-4286-8b96-5f44a73c049c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:06:34.238657 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.238636    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/449184db-c80d-4286-8b96-5f44a73c049c-kube-api-access-nzzfp" (OuterVolumeSpecName: "kube-api-access-nzzfp") pod "449184db-c80d-4286-8b96-5f44a73c049c" (UID: "449184db-c80d-4286-8b96-5f44a73c049c"). InnerVolumeSpecName "kube-api-access-nzzfp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:06:34.337728 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.337698    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nzzfp\" (UniqueName: \"kubernetes.io/projected/449184db-c80d-4286-8b96-5f44a73c049c-kube-api-access-nzzfp\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:34.337728 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.337725    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/449184db-c80d-4286-8b96-5f44a73c049c-maas-api-tls\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:34.337728 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.337734    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/449184db-c80d-4286-8b96-5f44a73c049c-openshift-service-ca\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:34.647818 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.647789    2563 generic.go:356] "Generic (PLEG): container finished" podID="449184db-c80d-4286-8b96-5f44a73c049c" containerID="cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05" exitCode=0
Jul 22 13:06:34.648002 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.647838    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"
Jul 22 13:06:34.648002 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.647838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp" event={"ID":"449184db-c80d-4286-8b96-5f44a73c049c","Type":"ContainerDied","Data":"cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05"}
Jul 22 13:06:34.648002 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.647976    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp" event={"ID":"449184db-c80d-4286-8b96-5f44a73c049c","Type":"ContainerDied","Data":"e775c01b1772ad425d8377609610fb864c3eaa3af3bebd718c23d5a880ba4a99"}
Jul 22 13:06:34.648383 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.648016    2563 scope.go:117] "RemoveContainer" containerID="cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05"
Jul 22 13:06:34.657968 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.657951    2563 scope.go:117] "RemoveContainer" containerID="cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05"
Jul 22 13:06:34.658231 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:06:34.658212    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05\": container with ID starting with cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05 not found: ID does not exist" containerID="cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05"
Jul 22 13:06:34.658279 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.658239    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05"} err="failed to get container status \"cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05\": rpc error: code = NotFound desc = could not find container \"cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05\": container with ID starting with cfad2d068daac4571ddec65e403127584fe11981490de3eae6bb698dbca76f05 not found: ID does not exist"
Jul 22 13:06:34.669138 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.669118    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"]
Jul 22 13:06:34.675002 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:34.674983    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7638e0e4-6db98cdd49-mcjtp"]
Jul 22 13:06:36.018132 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:36.018102    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="449184db-c80d-4286-8b96-5f44a73c049c" path="/var/lib/kubelet/pods/449184db-c80d-4286-8b96-5f44a73c049c/volumes"
Jul 22 13:06:38.628930 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:38.628896    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"]
Jul 22 13:06:38.629357 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:38.629155    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy" containerID="cri-o://76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33" gracePeriod=30
Jul 22 13:06:39.649160 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.649127    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"]
Jul 22 13:06:39.649681 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.649657    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="449184db-c80d-4286-8b96-5f44a73c049c" containerName="maas-api"
Jul 22 13:06:39.649681 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.649682    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="449184db-c80d-4286-8b96-5f44a73c049c" containerName="maas-api"
Jul 22 13:06:39.649921 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.649772    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="449184db-c80d-4286-8b96-5f44a73c049c" containerName="maas-api"
Jul 22 13:06:39.654221 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.654201    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.657533 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.657516    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-3147e54f-gw-openshift-default-dockercfg-qk7x2\""
Jul 22 13:06:39.668105 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.668081    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"]
Jul 22 13:06:39.784083 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-credential-socket\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784083 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784086    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istiod-ca-cert\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784307 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784159    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-data\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784307 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784213    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blhpw\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-kube-api-access-blhpw\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784307 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-podinfo\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784307 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-certs\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784307 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784306    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-token\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784493 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784327    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-envoy\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.784493 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.784358    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-socket\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885359 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-data\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885530 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885369    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blhpw\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-kube-api-access-blhpw\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885530 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885391    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-podinfo\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885530 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-certs\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885714 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885539    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-token\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885714 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-envoy\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885714 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-socket\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885714 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-credential-socket\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885948 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885727    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istiod-ca-cert\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885948 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885806    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-data\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.885948 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.885855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-certs\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.886088 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.886024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-socket\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.886088 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.886069    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-credential-socket\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.886368 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.886313    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istiod-ca-cert\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.887986 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.887960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-envoy\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.888105 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.888090    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-podinfo\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.894244 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.894217    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-token\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.894442 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.894422    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blhpw\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-kube-api-access-blhpw\") pod \"e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") " pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:39.964937 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:39.964908    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:40.105899 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.105769    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"]
Jul 22 13:06:40.108838 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:06:40.108811    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d42d95_53c3_4a5c_84e7_d44043d46ab6.slice/crio-9bbffdbadf2d3e2a63ec36243c86639210cd72639e9185ab11ceac03fccd34c3 WatchSource:0}: Error finding container 9bbffdbadf2d3e2a63ec36243c86639210cd72639e9185ab11ceac03fccd34c3: Status 404 returned error can't find the container with id 9bbffdbadf2d3e2a63ec36243c86639210cd72639e9185ab11ceac03fccd34c3
Jul 22 13:06:40.111113 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.111074    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:40.111177 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.111161    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:40.111224 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.111203    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:40.686685 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.686630    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" event={"ID":"34d42d95-53c3-4a5c-84e7-d44043d46ab6","Type":"ContainerStarted","Data":"fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9"}
Jul 22 13:06:40.687128 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.686733    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" event={"ID":"34d42d95-53c3-4a5c-84e7-d44043d46ab6","Type":"ContainerStarted","Data":"9bbffdbadf2d3e2a63ec36243c86639210cd72639e9185ab11ceac03fccd34c3"}
Jul 22 13:06:40.709862 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.709817    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" podStartSLOduration=1.7098032 podStartE2EDuration="1.7098032s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:06:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:06:40.70711285 +0000 UTC m=+1785.172010899" watchObservedRunningTime="2026-07-22 13:06:40.7098032 +0000 UTC m=+1785.174701249"
Jul 22 13:06:40.965852 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:40.965825    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:41.970206 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:41.970180    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:42.696065 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:42.696031    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:42.696931 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:42.696911    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:43.882473 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:43.882452    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:06:44.026028 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026002    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-certs\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026066    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-token\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026097    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-podinfo\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026116    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2gnd\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-kube-api-access-b2gnd\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026131    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-data\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026150    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-socket\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026461 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026229    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istiod-ca-cert\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026461 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026252    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:06:44.026461 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026269    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-credential-socket\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026461 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026315    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-envoy\") pod \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\" (UID: \"db71f851-1a96-4a82-b6c5-fa5a4d96ef87\") "
Jul 22 13:06:44.026667 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026462    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:06:44.026667 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026618    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.026667 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026624    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:06:44.026667 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026637    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.026667 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.026649    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:06:44.027124 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.027004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-data" (OuterVolumeSpecName: "istio-data") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:06:44.028414 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.028387    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:06:44.028600 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.028579    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:06:44.028979 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.028961    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-token" (OuterVolumeSpecName: "istio-token") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:06:44.028979 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.028961    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-kube-api-access-b2gnd" (OuterVolumeSpecName: "kube-api-access-b2gnd") pod "db71f851-1a96-4a82-b6c5-fa5a4d96ef87" (UID: "db71f851-1a96-4a82-b6c5-fa5a4d96ef87"). InnerVolumeSpecName "kube-api-access-b2gnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:06:44.127100 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.127078    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.127100 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.127098    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.127250 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.127109    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.127250 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.127117    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2gnd\" (UniqueName: \"kubernetes.io/projected/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-kube-api-access-b2gnd\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.127250 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.127128    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.127250 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.127137    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.127250 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.127145    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/db71f851-1a96-4a82-b6c5-fa5a4d96ef87-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:06:44.706392 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.706357    2563 generic.go:356] "Generic (PLEG): container finished" podID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerID="76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33" exitCode=0
Jul 22 13:06:44.706562 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.706406    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"
Jul 22 13:06:44.706562 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.706442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" event={"ID":"db71f851-1a96-4a82-b6c5-fa5a4d96ef87","Type":"ContainerDied","Data":"76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33"}
Jul 22 13:06:44.706562 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.706499    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4" event={"ID":"db71f851-1a96-4a82-b6c5-fa5a4d96ef87","Type":"ContainerDied","Data":"a3b7575f15e5b73816b67c87d9a61d826107af12a1b3ba17466a857b7b42a224"}
Jul 22 13:06:44.706562 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.706520    2563 scope.go:117] "RemoveContainer" containerID="76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33"
Jul 22 13:06:44.715605 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.715591    2563 scope.go:117] "RemoveContainer" containerID="fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8"
Jul 22 13:06:44.723943 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.723928    2563 scope.go:117] "RemoveContainer" containerID="76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33"
Jul 22 13:06:44.724234 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:06:44.724217    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33\": container with ID starting with 76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33 not found: ID does not exist" containerID="76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33"
Jul 22 13:06:44.724281 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.724244    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33"} err="failed to get container status \"76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33\": rpc error: code = NotFound desc = could not find container \"76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33\": container with ID starting with 76bde1df195aa29162b482dd5a5202a35b434ecfab0682d1b8cb37f14aaafc33 not found: ID does not exist"
Jul 22 13:06:44.724281 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.724262    2563 scope.go:117] "RemoveContainer" containerID="fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8"
Jul 22 13:06:44.724497 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:06:44.724480    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8\": container with ID starting with fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8 not found: ID does not exist" containerID="fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8"
Jul 22 13:06:44.724550 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.724502    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8"} err="failed to get container status \"fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8\": rpc error: code = NotFound desc = could not find container \"fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8\": container with ID starting with fd082cb18fcc87bc3a7576aeed7c37959fba8beb74cf9c7d335813c4d38a52f8 not found: ID does not exist"
Jul 22 13:06:44.731019 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.730999    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"]
Jul 22 13:06:44.734551 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:44.734533    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7638e0e4-openshift-default-78598499d7-h8sh4"]
Jul 22 13:06:46.023605 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:46.023571    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" path="/var/lib/kubelet/pods/db71f851-1a96-4a82-b6c5-fa5a4d96ef87/volumes"
Jul 22 13:06:56.664120 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:06:56.664095    2563 helpers.go:245] readString: Failed to read "/sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d42d95_53c3_4a5c_84e7_d44043d46ab6.slice/crio-conmon-fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9.scope/cpu.max": read /sys/fs/cgroup/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d42d95_53c3_4a5c_84e7_d44043d46ab6.slice/crio-conmon-fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9.scope/cpu.max: no such device
Jul 22 13:06:56.688904 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:06:56.688878    2563 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34d42d95_53c3_4a5c_84e7_d44043d46ab6.slice/crio-fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9.scope\": RecentStats: unable to find data in memory cache]"
Jul 22 13:06:56.765797 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:56.765761    2563 generic.go:356] "Generic (PLEG): container finished" podID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerID="fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9" exitCode=0
Jul 22 13:06:56.765945 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:56.765817    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" event={"ID":"34d42d95-53c3-4a5c-84e7-d44043d46ab6","Type":"ContainerDied","Data":"fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9"}
Jul 22 13:06:56.766185 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:56.766170    2563 scope.go:117] "RemoveContainer" containerID="fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9"
Jul 22 13:06:56.767190 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:56.767165    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:56.767241 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:56.767227    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:56.767280 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:56.767256    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:06:57.697037 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:57.697010    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:57.775098 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:57.775061    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" event={"ID":"34d42d95-53c3-4a5c-84e7-d44043d46ab6","Type":"ContainerStarted","Data":"9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae"}
Jul 22 13:06:57.967468 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:57.967357    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:06:57.967652 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:57.967498    2563 patch_prober.go:28] interesting pod/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 22 13:06:57.967652 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:57.967543    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 22 13:06:58.966619 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:58.966190    2563 patch_prober.go:28] interesting pod/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 22 13:06:58.966619 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:58.966249    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 22 13:06:59.965418 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:59.965381    2563 patch_prober.go:28] interesting pod/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 22 13:06:59.966086 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:06:59.966054    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 22 13:07:00.965691 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:00.965653    2563 patch_prober.go:28] interesting pod/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body=
Jul 22 13:07:00.966308 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:00.966278    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused"
Jul 22 13:07:01.969517 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:01.969482    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:07:01.969913 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:01.969742    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:07:01.970518 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:01.970499    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:07:22.409091 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:22.409060    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"]
Jul 22 13:07:22.409576 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:22.409333    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy" containerID="cri-o://9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae" gracePeriod=30
Jul 22 13:07:23.242072 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.242041    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"]
Jul 22 13:07:23.242437 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.242424    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy"
Jul 22 13:07:23.242484 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.242440    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy"
Jul 22 13:07:23.242484 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.242454    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy"
Jul 22 13:07:23.242484 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.242460    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy"
Jul 22 13:07:23.242590 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.242513    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy"
Jul 22 13:07:23.245857 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.245838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.248919 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.248897    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-bc6a4359-openshift-default-dockercfg-qzkp7\""
Jul 22 13:07:23.263005 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.262987    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"]
Jul 22 13:07:23.368975 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.368937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-token\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369143 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.368994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-envoy\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369143 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.369056    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/345ff36b-e0fc-4b2d-8e18-234b875c9844-istiod-ca-cert\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369143 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.369103    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-podinfo\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369143 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.369122    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-credential-socket\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369320 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.369144    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-data\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369320 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.369219    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-socket\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369320 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.369253    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-certs\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.369320 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.369275    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-kube-api-access-h4qxh\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470297 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-envoy\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470297 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470299    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/345ff36b-e0fc-4b2d-8e18-234b875c9844-istiod-ca-cert\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470806 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-podinfo\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470806 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-credential-socket\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470806 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-data\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470806 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470419    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-socket\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470806 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-certs\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470806 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470470    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-kube-api-access-h4qxh\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.470806 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470540    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-token\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.471194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.470822    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-socket\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.471194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.471010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-credential-socket\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.471194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.471115    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/345ff36b-e0fc-4b2d-8e18-234b875c9844-istiod-ca-cert\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.471194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.471170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-data\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.471194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.471189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-certs\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.472796 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.472777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-envoy\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.473024 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.473007    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-podinfo\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.481214 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.481178    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-token\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.481593 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.481569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-kube-api-access-h4qxh\") pod \"e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") " pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.557232 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.557151    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:23.697488 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.697464    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"]
Jul 22 13:07:23.699628 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:07:23.699600    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod345ff36b_e0fc_4b2d_8e18_234b875c9844.slice/crio-6774b34bfdb5db8006146ad9075ec08d20bb8f0c6647f07bf47eec768c7255e7 WatchSource:0}: Error finding container 6774b34bfdb5db8006146ad9075ec08d20bb8f0c6647f07bf47eec768c7255e7: Status 404 returned error can't find the container with id 6774b34bfdb5db8006146ad9075ec08d20bb8f0c6647f07bf47eec768c7255e7
Jul 22 13:07:23.701622 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.701584    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:07:23.701710 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.701650    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:07:23.701710 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.701679    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:07:23.904611 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.904527    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" event={"ID":"345ff36b-e0fc-4b2d-8e18-234b875c9844","Type":"ContainerStarted","Data":"e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978"}
Jul 22 13:07:23.904611 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.904589    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" event={"ID":"345ff36b-e0fc-4b2d-8e18-234b875c9844","Type":"ContainerStarted","Data":"6774b34bfdb5db8006146ad9075ec08d20bb8f0c6647f07bf47eec768c7255e7"}
Jul 22 13:07:23.934478 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:23.934430    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podStartSLOduration=0.934413379 podStartE2EDuration="934.413379ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:07:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:07:23.933102224 +0000 UTC m=+1828.398000274" watchObservedRunningTime="2026-07-22 13:07:23.934413379 +0000 UTC m=+1828.399311429"
Jul 22 13:07:24.557729 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:24.557696    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:24.559498 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:24.559473    2563 patch_prober.go:28] interesting pod/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 22 13:07:24.559612 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:24.559520    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 22 13:07:25.561377 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:25.561336    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:25.920590 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:25.920494    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:25.921602 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:25.921585    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:27.662264 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.662241    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:07:27.712761 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712720    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-token\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.712915 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712788    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-envoy\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.712915 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712832    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istiod-ca-cert\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.712915 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712860    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-credential-socket\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.712915 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712892    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-blhpw\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-kube-api-access-blhpw\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.713138 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712917    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-podinfo\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.713138 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712940    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-socket\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.713138 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.712985    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-data\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.713138 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.713090    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-certs\") pod \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\" (UID: \"34d42d95-53c3-4a5c-84e7-d44043d46ab6\") "
Jul 22 13:07:27.713345 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.713220    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:07:27.713705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.713519    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.713705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.713636    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:07:27.714074 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.713986    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:07:27.714225 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.714196    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:07:27.714501 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.714473    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-data" (OuterVolumeSpecName: "istio-data") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:07:27.715330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.715276    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:07:27.715429 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.715371    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-token" (OuterVolumeSpecName: "istio-token") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:07:27.715483 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.715422    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-kube-api-access-blhpw" (OuterVolumeSpecName: "kube-api-access-blhpw") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "kube-api-access-blhpw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:07:27.715597 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.715575    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "34d42d95-53c3-4a5c-84e7-d44043d46ab6" (UID: "34d42d95-53c3-4a5c-84e7-d44043d46ab6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:07:27.814969 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.814942    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.814969 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.814964    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.814969 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.814973    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.815173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.814980    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.815173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.814989    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-blhpw\" (UniqueName: \"kubernetes.io/projected/34d42d95-53c3-4a5c-84e7-d44043d46ab6-kube-api-access-blhpw\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.815173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.814998    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.815173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.815006    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.815173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.815015    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/34d42d95-53c3-4a5c-84e7-d44043d46ab6-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:07:27.931109 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.931076    2563 generic.go:356] "Generic (PLEG): container finished" podID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerID="9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae" exitCode=0
Jul 22 13:07:27.931259 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.931123    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"
Jul 22 13:07:27.931259 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.931165    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" event={"ID":"34d42d95-53c3-4a5c-84e7-d44043d46ab6","Type":"ContainerDied","Data":"9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae"}
Jul 22 13:07:27.931259 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.931219    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m" event={"ID":"34d42d95-53c3-4a5c-84e7-d44043d46ab6","Type":"ContainerDied","Data":"9bbffdbadf2d3e2a63ec36243c86639210cd72639e9185ab11ceac03fccd34c3"}
Jul 22 13:07:27.931259 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.931243    2563 scope.go:117] "RemoveContainer" containerID="9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae"
Jul 22 13:07:27.941325 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.941305    2563 scope.go:117] "RemoveContainer" containerID="fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9"
Jul 22 13:07:27.949024 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.949007    2563 scope.go:117] "RemoveContainer" containerID="9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae"
Jul 22 13:07:27.949319 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:07:27.949301    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae\": container with ID starting with 9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae not found: ID does not exist" containerID="9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae"
Jul 22 13:07:27.949372 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.949327    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae"} err="failed to get container status \"9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae\": rpc error: code = NotFound desc = could not find container \"9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae\": container with ID starting with 9d9d87d09799a068db97bab717b5f84d0986bdc4100c954f0add588f044710ae not found: ID does not exist"
Jul 22 13:07:27.949372 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.949345    2563 scope.go:117] "RemoveContainer" containerID="fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9"
Jul 22 13:07:27.949569 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:07:27.949554    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9\": container with ID starting with fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9 not found: ID does not exist" containerID="fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9"
Jul 22 13:07:27.949611 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.949574    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9"} err="failed to get container status \"fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9\": rpc error: code = NotFound desc = could not find container \"fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9\": container with ID starting with fb2396a6029b3b8a089658701a6862f20dcb6cf3ebdaa5d257228e5390db58e9 not found: ID does not exist"
Jul 22 13:07:27.954187 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.954166    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"]
Jul 22 13:07:27.958576 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:27.958555    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-3147e54f-gw-openshift-default-5694c5dc95-hfz4m"]
Jul 22 13:07:28.021550 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:28.021462    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" path="/var/lib/kubelet/pods/34d42d95-53c3-4a5c-84e7-d44043d46ab6/volumes"
Jul 22 13:07:40.002362 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:40.002279    2563 generic.go:356] "Generic (PLEG): container finished" podID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerID="e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978" exitCode=0
Jul 22 13:07:40.002362 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:40.002327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" event={"ID":"345ff36b-e0fc-4b2d-8e18-234b875c9844","Type":"ContainerDied","Data":"e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978"}
Jul 22 13:07:40.002908 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:40.002705    2563 scope.go:117] "RemoveContainer" containerID="e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978"
Jul 22 13:07:40.003876 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:40.003849    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:07:40.003943 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:40.003921    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:07:40.003988 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:40.003952    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:07:40.921383 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:40.921351    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:41.009304 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:41.009266    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" event={"ID":"345ff36b-e0fc-4b2d-8e18-234b875c9844","Type":"ContainerStarted","Data":"0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37"}
Jul 22 13:07:41.557577 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:41.557548    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:41.558589 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:41.558562    2563 patch_prober.go:28] interesting pod/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 22 13:07:41.558709 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:41.558618    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 22 13:07:42.558067 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:42.558023    2563 patch_prober.go:28] interesting pod/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 22 13:07:42.558648 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:42.558615    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 22 13:07:43.558029 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:43.557988    2563 patch_prober.go:28] interesting pod/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 22 13:07:43.558236 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:43.558060    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 22 13:07:44.558540 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:44.558502    2563 patch_prober.go:28] interesting pod/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 22 13:07:44.559157 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:44.559125    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 22 13:07:45.558273 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:45.557818    2563 patch_prober.go:28] interesting pod/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 22 13:07:45.558273 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:45.557883    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 22 13:07:46.557529 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:46.557487    2563 patch_prober.go:28] interesting pod/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 22 13:07:46.557937 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:46.557555    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 22 13:07:47.561590 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:47.561560    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:47.562075 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:47.561951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:07:47.562694 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:07:47.562675    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:08:07.348422 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:07.348389    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"]
Jul 22 13:08:07.349012 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:07.348635    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy" containerID="cri-o://0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37" gracePeriod=30
Jul 22 13:08:08.118684 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.118652    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"]
Jul 22 13:08:08.119110 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.119093    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy"
Jul 22 13:08:08.119199 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.119114    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy"
Jul 22 13:08:08.119199 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.119148    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy"
Jul 22 13:08:08.119199 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.119157    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy"
Jul 22 13:08:08.119355 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.119245    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy"
Jul 22 13:08:08.119355 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.119260    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="34d42d95-53c3-4a5c-84e7-d44043d46ab6" containerName="istio-proxy"
Jul 22 13:08:08.119355 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.119271    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="db71f851-1a96-4a82-b6c5-fa5a4d96ef87" containerName="istio-proxy"
Jul 22 13:08:08.121443 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.121423    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.124048 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.124031    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-0d907048-openshift-default-dockercfg-9phnp\""
Jul 22 13:08:08.140363 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.140341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"]
Jul 22 13:08:08.165597 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165572    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-envoy\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.165719 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-data\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.165719 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165620    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-socket\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.165719 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165640    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdq2g\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-kube-api-access-pdq2g\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.165866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165727    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d58cb54-65b4-44b8-bbce-25a916ef7979-istiod-ca-cert\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.165866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-certs\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.165866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-credential-socket\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.165866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-podinfo\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.166016 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.165915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-token\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.266598 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266558    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-token\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.266823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266620    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-envoy\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.266823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266648    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-data\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.266823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266673    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-socket\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.266823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266706    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdq2g\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-kube-api-access-pdq2g\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.266823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266736    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d58cb54-65b4-44b8-bbce-25a916ef7979-istiod-ca-cert\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.266823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-certs\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.267156 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-credential-socket\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.267156 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.266917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-podinfo\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.267156 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.267139    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-data\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.267307 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.267248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-socket\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.267369 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.267326    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-certs\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.267369 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.267337    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-credential-socket\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.267601 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.267582    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d58cb54-65b4-44b8-bbce-25a916ef7979-istiod-ca-cert\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.269230 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.269198    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-envoy\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.269425 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.269409    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-podinfo\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.275831 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.275810    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdq2g\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-kube-api-access-pdq2g\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.276043 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.276025    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-token\") pod \"e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") " pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.431592 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.431519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:08.571661 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.571636    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"]
Jul 22 13:08:08.573524 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:08:08.573493    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d58cb54_65b4_44b8_bbce_25a916ef7979.slice/crio-6af27521f72f631b0550133cc6d4a929fa097c77574ff0e18e436e9dc5cb0b2a WatchSource:0}: Error finding container 6af27521f72f631b0550133cc6d4a929fa097c77574ff0e18e436e9dc5cb0b2a: Status 404 returned error can't find the container with id 6af27521f72f631b0550133cc6d4a929fa097c77574ff0e18e436e9dc5cb0b2a
Jul 22 13:08:08.576044 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.576004    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:08:08.576164 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.576098    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:08:08.576164 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:08.576133    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:08:09.142559 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:09.142523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" event={"ID":"2d58cb54-65b4-44b8-bbce-25a916ef7979","Type":"ContainerStarted","Data":"71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e"}
Jul 22 13:08:09.142740 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:09.142585    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" event={"ID":"2d58cb54-65b4-44b8-bbce-25a916ef7979","Type":"ContainerStarted","Data":"6af27521f72f631b0550133cc6d4a929fa097c77574ff0e18e436e9dc5cb0b2a"}
Jul 22 13:08:09.162846 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:09.162802    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" podStartSLOduration=1.162786973 podStartE2EDuration="1.162786973s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:08:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:08:09.162529667 +0000 UTC m=+1873.627427715" watchObservedRunningTime="2026-07-22 13:08:09.162786973 +0000 UTC m=+1873.627685019"
Jul 22 13:08:09.432487 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:09.432406    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:09.437348 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:09.437328    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:10.147685 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:10.147653    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:10.148513 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:10.148487    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:12.602900 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.602877    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:08:12.708440 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708410    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-envoy\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708629 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708445    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-credential-socket\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708629 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708478    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-certs\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708629 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708522    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-token\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708629 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708569    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/345ff36b-e0fc-4b2d-8e18-234b875c9844-istiod-ca-cert\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708629 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708625    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-podinfo\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708648    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-data\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708689    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-socket\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708741    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-kube-api-access-h4qxh\") pod \"345ff36b-e0fc-4b2d-8e18-234b875c9844\" (UID: \"345ff36b-e0fc-4b2d-8e18-234b875c9844\") "
Jul 22 13:08:12.708936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708805    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:12.708936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.708859    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:12.709200 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.709034    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/345ff36b-e0fc-4b2d-8e18-234b875c9844-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:08:12.709200 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.709053    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:12.709200 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.709141    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.709200 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.709165    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.709394 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.709276    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/345ff36b-e0fc-4b2d-8e18-234b875c9844-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.709394 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.709351    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.709563 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.709502    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-data" (OuterVolumeSpecName: "istio-data") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:12.710898 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.710871    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:12.711279 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.711261    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-token" (OuterVolumeSpecName: "istio-token") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:08:12.711474 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.711452    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:08:12.711474 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.711462    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-kube-api-access-h4qxh" (OuterVolumeSpecName: "kube-api-access-h4qxh") pod "345ff36b-e0fc-4b2d-8e18-234b875c9844" (UID: "345ff36b-e0fc-4b2d-8e18-234b875c9844"). InnerVolumeSpecName "kube-api-access-h4qxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:08:12.809888 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.809855    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.809888 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.809879    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.809888 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.809889    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h4qxh\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-kube-api-access-h4qxh\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.810104 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.809899    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:12.810104 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:12.809907    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/345ff36b-e0fc-4b2d-8e18-234b875c9844-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:13.164068 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.163986    2563 generic.go:356] "Generic (PLEG): container finished" podID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerID="0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37" exitCode=0
Jul 22 13:08:13.164068 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.164033    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"
Jul 22 13:08:13.164301 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.164077    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" event={"ID":"345ff36b-e0fc-4b2d-8e18-234b875c9844","Type":"ContainerDied","Data":"0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37"}
Jul 22 13:08:13.164301 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.164139    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp" event={"ID":"345ff36b-e0fc-4b2d-8e18-234b875c9844","Type":"ContainerDied","Data":"6774b34bfdb5db8006146ad9075ec08d20bb8f0c6647f07bf47eec768c7255e7"}
Jul 22 13:08:13.164301 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.164170    2563 scope.go:117] "RemoveContainer" containerID="0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37"
Jul 22 13:08:13.174160 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.174145    2563 scope.go:117] "RemoveContainer" containerID="e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978"
Jul 22 13:08:13.181928 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.181910    2563 scope.go:117] "RemoveContainer" containerID="0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37"
Jul 22 13:08:13.182179 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:08:13.182160    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37\": container with ID starting with 0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37 not found: ID does not exist" containerID="0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37"
Jul 22 13:08:13.182222 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.182187    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37"} err="failed to get container status \"0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37\": rpc error: code = NotFound desc = could not find container \"0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37\": container with ID starting with 0a16c0329842b34929057a23c8150d226020dc91f33902044681f7ba0238ed37 not found: ID does not exist"
Jul 22 13:08:13.182222 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.182206    2563 scope.go:117] "RemoveContainer" containerID="e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978"
Jul 22 13:08:13.182396 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:08:13.182380    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978\": container with ID starting with e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978 not found: ID does not exist" containerID="e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978"
Jul 22 13:08:13.182437 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.182400    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978"} err="failed to get container status \"e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978\": rpc error: code = NotFound desc = could not find container \"e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978\": container with ID starting with e4fcc4aab047b40674b0569729d8754d4eabddeb4c581e1b6922812510154978 not found: ID does not exist"
Jul 22 13:08:13.187988 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.187956    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"]
Jul 22 13:08:13.192174 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:13.192156    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-bc6a4359-openshift-default-84f56b4669-tkrdp"]
Jul 22 13:08:14.020546 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:14.020511    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" path="/var/lib/kubelet/pods/345ff36b-e0fc-4b2d-8e18-234b875c9844/volumes"
Jul 22 13:08:22.211500 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:22.211468    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerID="71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e" exitCode=0
Jul 22 13:08:22.211950 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:22.211544    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" event={"ID":"2d58cb54-65b4-44b8-bbce-25a916ef7979","Type":"ContainerDied","Data":"71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e"}
Jul 22 13:08:22.211950 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:22.211928    2563 scope.go:117] "RemoveContainer" containerID="71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e"
Jul 22 13:08:22.212945 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:22.212918    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:08:22.213061 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:22.212998    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:08:22.213128 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:22.213056    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:08:23.218719 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:23.218682    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" event={"ID":"2d58cb54-65b4-44b8-bbce-25a916ef7979","Type":"ContainerStarted","Data":"9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206"}
Jul 22 13:08:23.432270 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:23.432234    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:23.433522 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:23.433492    2563 patch_prober.go:28] interesting pod/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:08:23.433637 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:23.433571    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 22 13:08:24.432051 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:24.432015    2563 patch_prober.go:28] interesting pod/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:08:24.432457 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:24.432076    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 22 13:08:25.147665 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:25.147635    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:25.432168 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:25.432091    2563 patch_prober.go:28] interesting pod/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:08:25.432168 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:25.432143    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 22 13:08:26.431965 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:26.431933    2563 patch_prober.go:28] interesting pod/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body=
Jul 22 13:08:26.432166 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:26.432110    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused"
Jul 22 13:08:27.436344 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:27.436308    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:27.437326 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:27.437307    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:50.611897 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:50.611860    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"]
Jul 22 13:08:50.612503 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:50.612176    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy" containerID="cri-o://9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206" gracePeriod=30
Jul 22 13:08:55.866344 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.866317    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:55.891553 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891474    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-envoy\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.891553 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891526    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-credential-socket\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.891725 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891590    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-socket\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.891725 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891629    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-data\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.891725 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891652    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdq2g\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-kube-api-access-pdq2g\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.891725 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891698    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-podinfo\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.891725 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-certs\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.892037 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d58cb54-65b4-44b8-bbce-25a916ef7979-istiod-ca-cert\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.892037 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891810    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-token\") pod \"2d58cb54-65b4-44b8-bbce-25a916ef7979\" (UID: \"2d58cb54-65b4-44b8-bbce-25a916ef7979\") "
Jul 22 13:08:55.892037 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891885    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:55.892037 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.891949    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:55.892252 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.892134    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:55.892252 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.892157    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.892252 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.892177    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.892451 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.892411    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d58cb54-65b4-44b8-bbce-25a916ef7979-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:08:55.892549 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.892525    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-data" (OuterVolumeSpecName: "istio-data") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:55.894182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.894158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-token" (OuterVolumeSpecName: "istio-token") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:08:55.894289 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.894260    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:08:55.894343 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.894295    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-kube-api-access-pdq2g" (OuterVolumeSpecName: "kube-api-access-pdq2g") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "kube-api-access-pdq2g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:08:55.894431 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.894408    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2d58cb54-65b4-44b8-bbce-25a916ef7979" (UID: "2d58cb54-65b4-44b8-bbce-25a916ef7979"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:08:55.993097 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.993068    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.993097 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.993095    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdq2g\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-kube-api-access-pdq2g\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.993263 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.993105    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.993263 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.993114    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.993263 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.993122    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2d58cb54-65b4-44b8-bbce-25a916ef7979-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.993263 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.993129    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:55.993263 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:55.993136    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2d58cb54-65b4-44b8-bbce-25a916ef7979-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:08:56.373152 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.373119    2563 generic.go:356] "Generic (PLEG): container finished" podID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerID="9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206" exitCode=0
Jul 22 13:08:56.373312 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.373165    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"
Jul 22 13:08:56.373312 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.373205    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" event={"ID":"2d58cb54-65b4-44b8-bbce-25a916ef7979","Type":"ContainerDied","Data":"9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206"}
Jul 22 13:08:56.373312 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.373260    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k" event={"ID":"2d58cb54-65b4-44b8-bbce-25a916ef7979","Type":"ContainerDied","Data":"6af27521f72f631b0550133cc6d4a929fa097c77574ff0e18e436e9dc5cb0b2a"}
Jul 22 13:08:56.373312 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.373290    2563 scope.go:117] "RemoveContainer" containerID="9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206"
Jul 22 13:08:56.381811 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.381794    2563 scope.go:117] "RemoveContainer" containerID="71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e"
Jul 22 13:08:56.389176 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.389159    2563 scope.go:117] "RemoveContainer" containerID="9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206"
Jul 22 13:08:56.389429 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:08:56.389412    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206\": container with ID starting with 9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206 not found: ID does not exist" containerID="9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206"
Jul 22 13:08:56.389472 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.389439    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206"} err="failed to get container status \"9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206\": rpc error: code = NotFound desc = could not find container \"9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206\": container with ID starting with 9a535fb72fc400bacd35de2c79a4be082d9d7fe7bd7daa4c6739751a732c4206 not found: ID does not exist"
Jul 22 13:08:56.389472 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.389460    2563 scope.go:117] "RemoveContainer" containerID="71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e"
Jul 22 13:08:56.389712 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:08:56.389694    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e\": container with ID starting with 71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e not found: ID does not exist" containerID="71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e"
Jul 22 13:08:56.389779 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.389720    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e"} err="failed to get container status \"71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e\": rpc error: code = NotFound desc = could not find container \"71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e\": container with ID starting with 71255020a92a2311de7392276f45f53b36b5319f603d1f4f341bfcfc965ec22e not found: ID does not exist"
Jul 22 13:08:56.392944 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.392924    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"]
Jul 22 13:08:56.396416 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:56.396396    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-0d907048-openshift-default-6b8799b7-nmh2k"]
Jul 22 13:08:58.018685 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:08:58.018649    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" path="/var/lib/kubelet/pods/2d58cb54-65b4-44b8-bbce-25a916ef7979/volumes"
Jul 22 13:10:29.691515 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691482    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"]
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691859    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691870    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691887    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691892    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691898    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691903    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691916    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691921    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691968    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691977    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691982    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d58cb54-65b4-44b8-bbce-25a916ef7979" containerName="istio-proxy"
Jul 22 13:10:29.691987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.691990    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="345ff36b-e0fc-4b2d-8e18-234b875c9844" containerName="istio-proxy"
Jul 22 13:10:29.693736 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.693720    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.698622 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.698600    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8c3b45b1-openshift-default-dockercfg-ltt7p\""
Jul 22 13:10:29.770096 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.770028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"]
Jul 22 13:10:29.817158 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a678e-d702-4f16-988a-a0598568274e-istiod-ca-cert\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817350 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-certs\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817350 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-credential-socket\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817350 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817271    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a678e-d702-4f16-988a-a0598568274e-istio-podinfo\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817350 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817304    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-data\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817527 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817356    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-socket\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817527 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jnrb\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-kube-api-access-7jnrb\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817527 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817417    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-istio-token\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.817527 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.817503    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-envoy\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918218 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-certs\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918218 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918222    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-credential-socket\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918242    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a678e-d702-4f16-988a-a0598568274e-istio-podinfo\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918270    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-data\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-socket\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jnrb\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-kube-api-access-7jnrb\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918464    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-istio-token\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-envoy\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918566    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a678e-d702-4f16-988a-a0598568274e-istiod-ca-cert\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-certs\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918945 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918653    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-data\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918945 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918678    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-credential-socket\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.918945 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.918843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-socket\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.919236 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.919215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a678e-d702-4f16-988a-a0598568274e-istiod-ca-cert\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.920844 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.920819    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-envoy\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.920948 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.920848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a678e-d702-4f16-988a-a0598568274e-istio-podinfo\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.936521 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.936495    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-istio-token\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:29.936628 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:29.936540    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jnrb\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-kube-api-access-7jnrb\") pod \"e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") " pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:30.004233 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.004166    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:30.152552 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.152460    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"]
Jul 22 13:10:30.154929 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:10:30.154902    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod435a678e_d702_4f16_988a_a0598568274e.slice/crio-b42bd35b33edc0626c852e3757d05b70c4d5bc30510f8cd29648ab466d909abd WatchSource:0}: Error finding container b42bd35b33edc0626c852e3757d05b70c4d5bc30510f8cd29648ab466d909abd: Status 404 returned error can't find the container with id b42bd35b33edc0626c852e3757d05b70c4d5bc30510f8cd29648ab466d909abd
Jul 22 13:10:30.157072 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.157036    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:10:30.157132 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.157099    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:10:30.157167 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.157131    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:10:30.801870 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.801835    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" event={"ID":"435a678e-d702-4f16-988a-a0598568274e","Type":"ContainerStarted","Data":"130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb"}
Jul 22 13:10:30.802272 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.801897    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" event={"ID":"435a678e-d702-4f16-988a-a0598568274e","Type":"ContainerStarted","Data":"b42bd35b33edc0626c852e3757d05b70c4d5bc30510f8cd29648ab466d909abd"}
Jul 22 13:10:30.827550 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:30.827494    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" podStartSLOduration=1.827479324 podStartE2EDuration="1.827479324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:10:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:10:30.826332902 +0000 UTC m=+2015.291230953" watchObservedRunningTime="2026-07-22 13:10:30.827479324 +0000 UTC m=+2015.292377374"
Jul 22 13:10:31.004933 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:31.004897    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:31.009333 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:31.009312    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:31.806893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:31.806866    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:31.807808 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:31.807791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:43.865866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:43.865733    2563 generic.go:356] "Generic (PLEG): container finished" podID="435a678e-d702-4f16-988a-a0598568274e" containerID="130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb" exitCode=0
Jul 22 13:10:43.865866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:43.865797    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" event={"ID":"435a678e-d702-4f16-988a-a0598568274e","Type":"ContainerDied","Data":"130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb"}
Jul 22 13:10:43.866418 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:43.866250    2563 scope.go:117] "RemoveContainer" containerID="130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb"
Jul 22 13:10:43.867314 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:43.867286    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:10:43.867414 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:43.867353    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:10:43.867414 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:43.867382    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:10:44.876480 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:44.876447    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" event={"ID":"435a678e-d702-4f16-988a-a0598568274e","Type":"ContainerStarted","Data":"805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6"}
Jul 22 13:10:45.005387 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:45.005348    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:45.006665 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:45.006637    2563 patch_prober.go:28] interesting pod/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 22 13:10:45.006802 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:45.006699    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 22 13:10:46.004842 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:46.004809    2563 patch_prober.go:28] interesting pod/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 22 13:10:46.005245 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:46.004872    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 22 13:10:46.807631 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:46.807592    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:47.005546 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:47.005508    2563 patch_prober.go:28] interesting pod/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 22 13:10:47.005963 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:47.005565    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 22 13:10:48.005130 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:48.005094    2563 patch_prober.go:28] interesting pod/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body=
Jul 22 13:10:48.005310 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:48.005149    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused"
Jul 22 13:10:49.008203 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:49.008176    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:10:49.009295 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:10:49.009274    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:11:05.995070 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:05.995040    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"]
Jul 22 13:11:05.995681 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:05.995268    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy" containerID="cri-o://805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6" gracePeriod=30
Jul 22 13:11:11.250648 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.250623    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:11:11.384726 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384648    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-envoy\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384726 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384684    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-data\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384726 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384714    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-certs\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384740    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a678e-d702-4f16-988a-a0598568274e-istiod-ca-cert\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384802    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-credential-socket\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384842    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-istio-token\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384880    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a678e-d702-4f16-988a-a0598568274e-istio-podinfo\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jnrb\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-kube-api-access-7jnrb\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.384990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384965    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-socket\") pod \"435a678e-d702-4f16-988a-a0598568274e\" (UID: \"435a678e-d702-4f16-988a-a0598568274e\") "
Jul 22 13:11:11.385294 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.384992    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:11:11.385294 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.385228    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:11:11.385294 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.385257    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.385468 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.385450    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:11:11.385604 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.385588    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/435a678e-d702-4f16-988a-a0598568274e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:11:11.385862 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.385845    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-data" (OuterVolumeSpecName: "istio-data") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:11:11.387674 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.387649    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/435a678e-d702-4f16-988a-a0598568274e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:11:11.387674 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.387657    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-istio-token" (OuterVolumeSpecName: "istio-token") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:11:11.387809 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.387690    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:11:11.387809 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.387701    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-kube-api-access-7jnrb" (OuterVolumeSpecName: "kube-api-access-7jnrb") pod "435a678e-d702-4f16-988a-a0598568274e" (UID: "435a678e-d702-4f16-988a-a0598568274e"). InnerVolumeSpecName "kube-api-access-7jnrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:11:11.485856 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485822    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.485856 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485851    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.486059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485863    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/435a678e-d702-4f16-988a-a0598568274e-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.486059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485875    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.486059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485887    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.486059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485898    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/435a678e-d702-4f16-988a-a0598568274e-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.486059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485908    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jnrb\" (UniqueName: \"kubernetes.io/projected/435a678e-d702-4f16-988a-a0598568274e-kube-api-access-7jnrb\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:11.486059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:11.485922    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/435a678e-d702-4f16-988a-a0598568274e-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:11:12.001950 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.001918    2563 generic.go:356] "Generic (PLEG): container finished" podID="435a678e-d702-4f16-988a-a0598568274e" containerID="805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6" exitCode=0
Jul 22 13:11:12.001950 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.001954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" event={"ID":"435a678e-d702-4f16-988a-a0598568274e","Type":"ContainerDied","Data":"805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6"}
Jul 22 13:11:12.002212 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.001970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"
Jul 22 13:11:12.002212 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.001996    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j" event={"ID":"435a678e-d702-4f16-988a-a0598568274e","Type":"ContainerDied","Data":"b42bd35b33edc0626c852e3757d05b70c4d5bc30510f8cd29648ab466d909abd"}
Jul 22 13:11:12.002212 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.002017    2563 scope.go:117] "RemoveContainer" containerID="805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6"
Jul 22 13:11:12.013246 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.013225    2563 scope.go:117] "RemoveContainer" containerID="130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb"
Jul 22 13:11:12.021653 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.021636    2563 scope.go:117] "RemoveContainer" containerID="805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6"
Jul 22 13:11:12.021959 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:11:12.021940    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6\": container with ID starting with 805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6 not found: ID does not exist" containerID="805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6"
Jul 22 13:11:12.022016 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.021968    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6"} err="failed to get container status \"805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6\": rpc error: code = NotFound desc = could not find container \"805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6\": container with ID starting with 805c482a8a278382ab88d230ec51d070a210e62e83fa719b56a63288045df7f6 not found: ID does not exist"
Jul 22 13:11:12.022016 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.021986    2563 scope.go:117] "RemoveContainer" containerID="130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb"
Jul 22 13:11:12.022204 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:11:12.022183    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb\": container with ID starting with 130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb not found: ID does not exist" containerID="130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb"
Jul 22 13:11:12.022255 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.022213    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb"} err="failed to get container status \"130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb\": rpc error: code = NotFound desc = could not find container \"130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb\": container with ID starting with 130e92d028ed83c51feb6938740da3370c75b49985ef034315d9f3696da8c4cb not found: ID does not exist"
Jul 22 13:11:12.035857 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.035832    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"]
Jul 22 13:11:12.041639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:12.041617    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8c3b45b1-openshift-default-5d8d685d87-hcv9j"]
Jul 22 13:11:14.018210 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:14.018175    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="435a678e-d702-4f16-988a-a0598568274e" path="/var/lib/kubelet/pods/435a678e-d702-4f16-988a-a0598568274e/volumes"
Jul 22 13:11:26.180940 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.180857    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"]
Jul 22 13:11:26.181290 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.181221    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy"
Jul 22 13:11:26.181290 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.181234    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy"
Jul 22 13:11:26.181361 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.181318    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy"
Jul 22 13:11:26.181361 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.181325    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy"
Jul 22 13:11:26.184276 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.184259    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.186931 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.186908    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-1cd4d457-openshift-default-dockercfg-lw6dd\""
Jul 22 13:11:26.198679 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.198661    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"]
Jul 22 13:11:26.317920 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.317883    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istiod-ca-cert\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318106 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.317928    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-podinfo\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318106 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.318011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9snm\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-kube-api-access-f9snm\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318106 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.318052    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-envoy\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318106 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.318073    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-certs\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318273 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.318160    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-credential-socket\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318273 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.318188    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-data\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318273 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.318211    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-token\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.318273 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.318242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-socket\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419232 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419197    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-socket\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419431 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419255    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istiod-ca-cert\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419431 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-podinfo\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419431 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9snm\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-kube-api-access-f9snm\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-envoy\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-certs\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-credential-socket\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419570    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-data\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-token\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419889 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419647    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-socket\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.419950 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419899    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-certs\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.420004 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419976    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-data\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.420004 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.419966    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-credential-socket\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.420104 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.420083    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istiod-ca-cert\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.421938 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.421912    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-envoy\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.422069 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.422053    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-podinfo\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.428951 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.428916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-token\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.429374 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.429352    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9snm\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-kube-api-access-f9snm\") pod \"e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") " pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.494544 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.494519    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:26.632859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.632826    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"]
Jul 22 13:11:26.636513 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:11:26.636481    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5992069b_cb4b_4a4e_a078_fb28c6fb1bcf.slice/crio-43f76d73e0bfe6ee797cdfc3cb69f5c84eadc5ad0714a8fa00c570686142efd2 WatchSource:0}: Error finding container 43f76d73e0bfe6ee797cdfc3cb69f5c84eadc5ad0714a8fa00c570686142efd2: Status 404 returned error can't find the container with id 43f76d73e0bfe6ee797cdfc3cb69f5c84eadc5ad0714a8fa00c570686142efd2
Jul 22 13:11:26.638727 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.638698    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:11:26.638817 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.638782    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:11:26.638817 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:26.638811    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:11:27.071038 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:27.070993    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" event={"ID":"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf","Type":"ContainerStarted","Data":"bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259"}
Jul 22 13:11:27.071222 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:27.071100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" event={"ID":"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf","Type":"ContainerStarted","Data":"43f76d73e0bfe6ee797cdfc3cb69f5c84eadc5ad0714a8fa00c570686142efd2"}
Jul 22 13:11:27.094844 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:27.094785    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podStartSLOduration=1.09473544 podStartE2EDuration="1.09473544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:11:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:11:27.093849374 +0000 UTC m=+2071.558747424" watchObservedRunningTime="2026-07-22 13:11:27.09473544 +0000 UTC m=+2071.559633489"
Jul 22 13:11:27.495332 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:27.495300    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:27.500335 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:27.500313    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:28.075500 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:28.075463    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:28.076605 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:28.076581    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:43.077687 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.077650    2563 patch_prober.go:28] interesting pod/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 22 13:11:43.078123 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.077721    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 22 13:11:43.147893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.147862    2563 generic.go:356] "Generic (PLEG): container finished" podID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerID="bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259" exitCode=0
Jul 22 13:11:43.148053 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.147908    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" event={"ID":"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf","Type":"ContainerDied","Data":"bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259"}
Jul 22 13:11:43.148289 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.148274    2563 scope.go:117] "RemoveContainer" containerID="bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259"
Jul 22 13:11:43.149269 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.149236    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:11:43.149383 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.149305    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:11:43.149383 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:43.149341    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:11:44.157708 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:44.157660    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" event={"ID":"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf","Type":"ContainerStarted","Data":"abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff"}
Jul 22 13:11:44.495721 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:44.495679    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:44.495934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:44.495919    2563 patch_prober.go:28] interesting pod/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 22 13:11:44.495991 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:44.495964    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 22 13:11:45.495317 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:45.495278    2563 patch_prober.go:28] interesting pod/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 22 13:11:45.495710 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:45.495339    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 22 13:11:46.495516 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:46.495486    2563 patch_prober.go:28] interesting pod/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 22 13:11:46.495944 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:46.495535    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 22 13:11:47.495000 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:47.494963    2563 patch_prober.go:28] interesting pod/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body=
Jul 22 13:11:47.495218 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:47.495029    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused"
Jul 22 13:11:48.499838 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:48.499808    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:48.500266 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:48.500100    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:11:48.501016 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:11:48.500998    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:12:08.729899 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.729866    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"]
Jul 22 13:12:08.730258 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.730226    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy"
Jul 22 13:12:08.730258 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.730237    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="435a678e-d702-4f16-988a-a0598568274e" containerName="istio-proxy"
Jul 22 13:12:08.732358 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.732339    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:08.736200 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.736176    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1cd4d457\"/\"kube-root-ca.crt\""
Jul 22 13:12:08.736329 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.736180    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1cd4d457\"/\"e2e-lifecycle-model-1cd4d457-kserve-self-signed-certs\""
Jul 22 13:12:08.736329 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.736269    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-1cd4d457\"/\"default-dockercfg-79r55\""
Jul 22 13:12:08.736455 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.736185    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-1cd4d457\"/\"openshift-service-ca.crt\""
Jul 22 13:12:08.743942 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.743919    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"]
Jul 22 13:12:08.901706 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.901671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-dshm\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:08.901914 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.901718    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-model-cache\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:08.901914 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.901773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxhrw\" (UniqueName: \"kubernetes.io/projected/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-kube-api-access-lxhrw\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:08.901914 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.901880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-home\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:08.902060 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.901943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tmp-dir\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:08.902060 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:08.901973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tls-certs\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.002895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.002793    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-home\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.002895 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.002882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tmp-dir\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.003133 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.002908    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tls-certs\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.003133 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.002942    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-dshm\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.003133 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.003078    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-model-cache\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.003133 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.003106    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxhrw\" (UniqueName: \"kubernetes.io/projected/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-kube-api-access-lxhrw\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.003341 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.003259    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-home\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.003341 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.003285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tmp-dir\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.003444 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.003425    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-model-cache\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.005423 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.005401    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-dshm\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.005576 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.005558    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tls-certs\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.012199 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.012180    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxhrw\" (UniqueName: \"kubernetes.io/projected/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-kube-api-access-lxhrw\") pod \"e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") " pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.042388 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.042367    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:09.179018 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.178975    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"]
Jul 22 13:12:09.181468 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:12:09.181439    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18813b02_d1ff_40b3_8be1_d28fbdd89bfa.slice/crio-1bc98dc867fcaa0723ae7651ee4cc0d8e7718393ec66fb95dfe9a01e9839e520 WatchSource:0}: Error finding container 1bc98dc867fcaa0723ae7651ee4cc0d8e7718393ec66fb95dfe9a01e9839e520: Status 404 returned error can't find the container with id 1bc98dc867fcaa0723ae7651ee4cc0d8e7718393ec66fb95dfe9a01e9839e520
Jul 22 13:12:09.283930 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.283856    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn" event={"ID":"18813b02-d1ff-40b3-8be1-d28fbdd89bfa","Type":"ContainerStarted","Data":"4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82"}
Jul 22 13:12:09.283930 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.283901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn" event={"ID":"18813b02-d1ff-40b3-8be1-d28fbdd89bfa","Type":"ContainerStarted","Data":"1bc98dc867fcaa0723ae7651ee4cc0d8e7718393ec66fb95dfe9a01e9839e520"}
Jul 22 13:12:09.301050 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:09.300982    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn" podStartSLOduration=1.300966377 podStartE2EDuration="1.300966377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:12:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:12:09.299721572 +0000 UTC m=+2113.764619626" watchObservedRunningTime="2026-07-22 13:12:09.300966377 +0000 UTC m=+2113.765864426"
Jul 22 13:12:19.042985 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:19.042951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:19.043474 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:19.043175    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:19.055267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:19.055235    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:12:19.340286 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:12:19.340194    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:13:05.025158 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:05.025071    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"]
Jul 22 13:13:20.045687 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.045650    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn" podUID="18813b02-d1ff-40b3-8be1-d28fbdd89bfa" containerName="main" containerID="cri-o://4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82" gracePeriod=45
Jul 22 13:13:20.298594 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.298535    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:13:20.322929 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.322904    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-home\") pod \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") "
Jul 22 13:13:20.323076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.322977    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tmp-dir\") pod \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") "
Jul 22 13:13:20.323076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.322999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tls-certs\") pod \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") "
Jul 22 13:13:20.323076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323037    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-model-cache\") pod \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") "
Jul 22 13:13:20.323076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323061    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxhrw\" (UniqueName: \"kubernetes.io/projected/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-kube-api-access-lxhrw\") pod \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") "
Jul 22 13:13:20.323304 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323088    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-dshm\") pod \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\" (UID: \"18813b02-d1ff-40b3-8be1-d28fbdd89bfa\") "
Jul 22 13:13:20.323304 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323180    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-home" (OuterVolumeSpecName: "home") pod "18813b02-d1ff-40b3-8be1-d28fbdd89bfa" (UID: "18813b02-d1ff-40b3-8be1-d28fbdd89bfa"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:20.323304 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323207    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18813b02-d1ff-40b3-8be1-d28fbdd89bfa" (UID: "18813b02-d1ff-40b3-8be1-d28fbdd89bfa"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:20.323439 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-model-cache" (OuterVolumeSpecName: "model-cache") pod "18813b02-d1ff-40b3-8be1-d28fbdd89bfa" (UID: "18813b02-d1ff-40b3-8be1-d28fbdd89bfa"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:20.323439 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323401    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:20.323439 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.323420    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:20.325490 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.325460    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18813b02-d1ff-40b3-8be1-d28fbdd89bfa" (UID: "18813b02-d1ff-40b3-8be1-d28fbdd89bfa"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:13:20.326057 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.326033    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-dshm" (OuterVolumeSpecName: "dshm") pod "18813b02-d1ff-40b3-8be1-d28fbdd89bfa" (UID: "18813b02-d1ff-40b3-8be1-d28fbdd89bfa"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:20.326183 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.326056    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-kube-api-access-lxhrw" (OuterVolumeSpecName: "kube-api-access-lxhrw") pod "18813b02-d1ff-40b3-8be1-d28fbdd89bfa" (UID: "18813b02-d1ff-40b3-8be1-d28fbdd89bfa"). InnerVolumeSpecName "kube-api-access-lxhrw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:13:20.424120 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.424095    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:20.424120 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.424119    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxhrw\" (UniqueName: \"kubernetes.io/projected/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-kube-api-access-lxhrw\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:20.424291 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.424127    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:20.424291 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.424137    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18813b02-d1ff-40b3-8be1-d28fbdd89bfa-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:20.617196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.617112    2563 generic.go:356] "Generic (PLEG): container finished" podID="18813b02-d1ff-40b3-8be1-d28fbdd89bfa" containerID="4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82" exitCode=0
Jul 22 13:13:20.617196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.617155    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"
Jul 22 13:13:20.617196 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.617180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn" event={"ID":"18813b02-d1ff-40b3-8be1-d28fbdd89bfa","Type":"ContainerDied","Data":"4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82"}
Jul 22 13:13:20.617425 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.617211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn" event={"ID":"18813b02-d1ff-40b3-8be1-d28fbdd89bfa","Type":"ContainerDied","Data":"1bc98dc867fcaa0723ae7651ee4cc0d8e7718393ec66fb95dfe9a01e9839e520"}
Jul 22 13:13:20.617425 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.617230    2563 scope.go:117] "RemoveContainer" containerID="4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82"
Jul 22 13:13:20.627299 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.627270    2563 scope.go:117] "RemoveContainer" containerID="4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82"
Jul 22 13:13:20.631485 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:13:20.631455    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82\": container with ID starting with 4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82 not found: ID does not exist" containerID="4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82"
Jul 22 13:13:20.631603 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.631497    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82"} err="failed to get container status \"4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82\": rpc error: code = NotFound desc = could not find container \"4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82\": container with ID starting with 4d3cb7e8582a6976c8ac9655526e53274bb8c907c9ec60bf74496b749f2b0f82 not found: ID does not exist"
Jul 22 13:13:20.643293 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.643264    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"]
Jul 22 13:13:20.649555 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:20.649535    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-1cd4d457/e2e-lifecycle-model-1cd4d457-kserve-768f5869c9-95bcn"]
Jul 22 13:13:22.018120 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:22.018089    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18813b02-d1ff-40b3-8be1-d28fbdd89bfa" path="/var/lib/kubelet/pods/18813b02-d1ff-40b3-8be1-d28fbdd89bfa/volumes"
Jul 22 13:13:48.052251 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:48.052217    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"]
Jul 22 13:13:48.052727 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:48.052441    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy" containerID="cri-o://abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff" gracePeriod=30
Jul 22 13:13:48.502517 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:48.502477    2563 patch_prober.go:28] interesting pod/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 13:13:48.502683 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:48.502534    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:13:53.301590 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.301565    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:13:53.401903 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.401830    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-podinfo\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.401903 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.401859    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-certs\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.401903 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.401896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-data\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.402181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.401914    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-token\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.402181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.401942    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-credential-socket\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.402181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.401978    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istiod-ca-cert\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.402181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402009    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-socket\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.402181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402052    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f9snm\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-kube-api-access-f9snm\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.402181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402128    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-envoy\") pod \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\" (UID: \"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf\") "
Jul 22 13:13:53.402181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:53.402583 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402250    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:53.402583 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402385    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:53.402583 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402527    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.402583 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402548    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.402583 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402563    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.402851 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.402617    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:13:53.403193 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.403162    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-data" (OuterVolumeSpecName: "istio-data") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:53.404515 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.404480    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:13:53.404515 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.404501    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-kube-api-access-f9snm" (OuterVolumeSpecName: "kube-api-access-f9snm") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "kube-api-access-f9snm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:13:53.404724 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.404705    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:13:53.404803 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.404780    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-token" (OuterVolumeSpecName: "istio-token") pod "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" (UID: "5992069b-cb4b-4a4e-a078-fb28c6fb1bcf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:13:53.503194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.503159    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.503194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.503193    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.503194 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.503204    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.503389 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.503212    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.503389 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.503220    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f9snm\" (UniqueName: \"kubernetes.io/projected/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-kube-api-access-f9snm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.503389 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.503230    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:13:53.775419 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.775384    2563 generic.go:356] "Generic (PLEG): container finished" podID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerID="abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff" exitCode=0
Jul 22 13:13:53.775574 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.775439    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"
Jul 22 13:13:53.775574 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.775469    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" event={"ID":"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf","Type":"ContainerDied","Data":"abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff"}
Jul 22 13:13:53.775574 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.775523    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v" event={"ID":"5992069b-cb4b-4a4e-a078-fb28c6fb1bcf","Type":"ContainerDied","Data":"43f76d73e0bfe6ee797cdfc3cb69f5c84eadc5ad0714a8fa00c570686142efd2"}
Jul 22 13:13:53.775574 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.775546    2563 scope.go:117] "RemoveContainer" containerID="abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff"
Jul 22 13:13:53.786504 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.786484    2563 scope.go:117] "RemoveContainer" containerID="bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259"
Jul 22 13:13:53.794599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.794580    2563 scope.go:117] "RemoveContainer" containerID="abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff"
Jul 22 13:13:53.794877 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:13:53.794859    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff\": container with ID starting with abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff not found: ID does not exist" containerID="abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff"
Jul 22 13:13:53.794935 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.794885    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff"} err="failed to get container status \"abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff\": rpc error: code = NotFound desc = could not find container \"abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff\": container with ID starting with abe3fc665ad7aeb17d5f0c5ada8238babe55fae6ac7aa791d61eb481b7afcdff not found: ID does not exist"
Jul 22 13:13:53.794935 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.794902    2563 scope.go:117] "RemoveContainer" containerID="bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259"
Jul 22 13:13:53.795134 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:13:53.795116    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259\": container with ID starting with bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259 not found: ID does not exist" containerID="bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259"
Jul 22 13:13:53.795176 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.795142    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259"} err="failed to get container status \"bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259\": rpc error: code = NotFound desc = could not find container \"bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259\": container with ID starting with bef51bfccf6e1d925a730ec04b481d263fec253b0163d051c55474d7a5aad259 not found: ID does not exist"
Jul 22 13:13:53.800214 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.800190    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"]
Jul 22 13:13:53.804543 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:53.804521    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-1cd4d457-openshift-default-7d7bd8cb7f-scc9v"]
Jul 22 13:13:54.024137 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:13:54.024093    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" path="/var/lib/kubelet/pods/5992069b-cb4b-4a4e-a078-fb28c6fb1bcf/volumes"
Jul 22 13:14:43.787918 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.787886    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"]
Jul 22 13:14:43.788331 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.788259    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy"
Jul 22 13:14:43.788331 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.788271    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy"
Jul 22 13:14:43.788331 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.788292    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18813b02-d1ff-40b3-8be1-d28fbdd89bfa" containerName="main"
Jul 22 13:14:43.788331 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.788298    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="18813b02-d1ff-40b3-8be1-d28fbdd89bfa" containerName="main"
Jul 22 13:14:43.788465 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.788352    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="18813b02-d1ff-40b3-8be1-d28fbdd89bfa" containerName="main"
Jul 22 13:14:43.788465 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.788361    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy"
Jul 22 13:14:43.791227 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.791211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.793829 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.793807    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-6b8164-openshift-default-dockercfg-kzk66\""
Jul 22 13:14:43.808452 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.808426    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"]
Jul 22 13:14:43.935533 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-token\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935543    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgxhg\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-kube-api-access-rgxhg\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-credential-socket\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935649    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-certs\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-podinfo\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935696    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-envoy\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935906 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935739    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-data\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935906 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf478d0f-0a5a-498e-9589-e2b4ead49452-istiod-ca-cert\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:43.935906 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:43.935816    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-socket\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036481    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-data\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf478d0f-0a5a-498e-9589-e2b4ead49452-istiod-ca-cert\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036724 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-socket\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036789 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036728    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-token\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036835 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036791    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgxhg\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-kube-api-access-rgxhg\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036875 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036849    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-credential-socket\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036928 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-data\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.036928 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-certs\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.037030 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-podinfo\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.037030 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-envoy\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.037030 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.036968    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-socket\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.037171 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.037120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-credential-socket\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.037251 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.037202    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-certs\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.037313 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.037268    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf478d0f-0a5a-498e-9589-e2b4ead49452-istiod-ca-cert\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.039363 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.039306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-envoy\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.039533 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.039512    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-podinfo\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.045364 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.045335    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-token\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.045537 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.045520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgxhg\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-kube-api-access-rgxhg\") pod \"e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") " pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.102117 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.102088    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:44.244992 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.244967    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"]
Jul 22 13:14:44.247047 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:14:44.246941    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf478d0f_0a5a_498e_9589_e2b4ead49452.slice/crio-a727c5f286c13bbc9260719d423b213c99d42fddad0f0132d23828ccc46fca75 WatchSource:0}: Error finding container a727c5f286c13bbc9260719d423b213c99d42fddad0f0132d23828ccc46fca75: Status 404 returned error can't find the container with id a727c5f286c13bbc9260719d423b213c99d42fddad0f0132d23828ccc46fca75
Jul 22 13:14:44.249316 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.249283    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:14:44.249427 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.249360    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:14:44.249427 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:44.249405    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:14:45.012142 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:45.012103    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" event={"ID":"cf478d0f-0a5a-498e-9589-e2b4ead49452","Type":"ContainerStarted","Data":"4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36"}
Jul 22 13:14:45.012545 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:45.012180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" event={"ID":"cf478d0f-0a5a-498e-9589-e2b4ead49452","Type":"ContainerStarted","Data":"a727c5f286c13bbc9260719d423b213c99d42fddad0f0132d23828ccc46fca75"}
Jul 22 13:14:45.034539 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:45.034489    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" podStartSLOduration=2.034473382 podStartE2EDuration="2.034473382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:14:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:14:45.032321974 +0000 UTC m=+2269.497220026" watchObservedRunningTime="2026-07-22 13:14:45.034473382 +0000 UTC m=+2269.499371432"
Jul 22 13:14:45.103240 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:45.103207    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:45.108216 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:45.108189    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:46.019182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:46.019156    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:46.019569 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:46.019220    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:14:52.165573 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.165539    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"]
Jul 22 13:14:52.166076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.165958    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy"
Jul 22 13:14:52.166076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.165971    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy"
Jul 22 13:14:52.166076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.166033    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5992069b-cb4b-4a4e-a078-fb28c6fb1bcf" containerName="istio-proxy"
Jul 22 13:14:52.169246 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.169216    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.171954 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.171929    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-c5ea07-openshift-default-dockercfg-jp6hf\""
Jul 22 13:14:52.181419 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.181397    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"]
Jul 22 13:14:52.313886 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.313846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-podinfo\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.313886 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.313889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-certs\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.314097 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.313959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-envoy\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.314097 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.314019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-token\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.314097 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.314050    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-data\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.314097 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.314076    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istiod-ca-cert\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.314229 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.314135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-credential-socket\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.314229 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.314177    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-socket\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.314229 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.314193    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mv9s\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-kube-api-access-9mv9s\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415466 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415434    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-socket\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415466 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415466    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mv9s\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-kube-api-access-9mv9s\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415502    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-podinfo\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-certs\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-envoy\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415594    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-token\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-data\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istiod-ca-cert\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.415712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415705    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-credential-socket\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.416107 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-socket\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.416107 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.415977    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-certs\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.416107 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.416089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-data\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.416213 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.416171    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-credential-socket\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.416429 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.416410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istiod-ca-cert\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.418052 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.418031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-envoy\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.418341 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.418321    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-podinfo\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.423941 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.423920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-token\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.424073 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.424056    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mv9s\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-kube-api-access-9mv9s\") pod \"e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") " pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.481104 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.481073    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:52.625695 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.625663    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"]
Jul 22 13:14:52.628851 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:14:52.628812    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3a0b84e_b31e_4b0e_a840_a7744c784c21.slice/crio-5f5a3e97395923b12ff691c090d9383155c7d971aa846ce12e2c20a2ae1fe064 WatchSource:0}: Error finding container 5f5a3e97395923b12ff691c090d9383155c7d971aa846ce12e2c20a2ae1fe064: Status 404 returned error can't find the container with id 5f5a3e97395923b12ff691c090d9383155c7d971aa846ce12e2c20a2ae1fe064
Jul 22 13:14:52.631027 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.631000    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:14:52.631101 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.631062    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:14:52.631101 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:52.631087    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:14:53.052297 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:53.052259    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" event={"ID":"d3a0b84e-b31e-4b0e-a840-a7744c784c21","Type":"ContainerStarted","Data":"9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5"}
Jul 22 13:14:53.052536 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:53.052327    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" event={"ID":"d3a0b84e-b31e-4b0e-a840-a7744c784c21","Type":"ContainerStarted","Data":"5f5a3e97395923b12ff691c090d9383155c7d971aa846ce12e2c20a2ae1fe064"}
Jul 22 13:14:53.072274 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:53.072222    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" podStartSLOduration=1.072207055 podStartE2EDuration="1.072207055s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:14:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:14:53.071453072 +0000 UTC m=+2277.536351120" watchObservedRunningTime="2026-07-22 13:14:53.072207055 +0000 UTC m=+2277.537105252"
Jul 22 13:14:53.481993 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:53.481964    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:53.486996 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:53.486972    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:54.060680 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:54.060612    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:14:54.065543 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:14:54.065511    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:15:00.096311 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:00.096276    2563 generic.go:356] "Generic (PLEG): container finished" podID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerID="4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36" exitCode=0
Jul 22 13:15:00.096311 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:00.096343    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" event={"ID":"cf478d0f-0a5a-498e-9589-e2b4ead49452","Type":"ContainerDied","Data":"4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36"}
Jul 22 13:15:00.096870 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:00.096733    2563 scope.go:117] "RemoveContainer" containerID="4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36"
Jul 22 13:15:00.097862 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:00.097836    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:15:00.097943 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:00.097904    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:15:00.097943 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:00.097934    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:15:01.017826 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:01.017788    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:15:01.104081 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:01.104047    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" event={"ID":"cf478d0f-0a5a-498e-9589-e2b4ead49452","Type":"ContainerStarted","Data":"7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9"}
Jul 22 13:15:02.102836 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:02.102798    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:15:02.104140 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:02.104106    2563 patch_prober.go:28] interesting pod/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 22 13:15:02.104508 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:02.104183    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 22 13:15:03.103255 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:03.103207    2563 patch_prober.go:28] interesting pod/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 22 13:15:03.103485 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:03.103281    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 22 13:15:04.102868 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:04.102826    2563 patch_prober.go:28] interesting pod/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 22 13:15:04.103353 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:04.102890    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 22 13:15:05.103409 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.103374    2563 patch_prober.go:28] interesting pod/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body=
Jul 22 13:15:05.103916 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.103887    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused"
Jul 22 13:15:05.335514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.335482    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"]
Jul 22 13:15:05.338268 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.338247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.342037 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.342013    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6b8164\"/\"default-dockercfg-8sr7w\""
Jul 22 13:15:05.342170 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.342075    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6b8164\"/\"openshift-service-ca.crt\""
Jul 22 13:15:05.342170 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.342020    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-6b8164\"/\"test-model-6b8164-kserve-self-signed-certs\""
Jul 22 13:15:05.342170 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.342019    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-6b8164\"/\"kube-root-ca.crt\""
Jul 22 13:15:05.348301 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.348280    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"]
Jul 22 13:15:05.441513 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.441486    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tls-certs\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.441661 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.441532    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-home\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.441706 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.441673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-model-cache\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.441742 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.441705    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-dshm\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.441742 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.441730    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tmp-dir\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.441828 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.441784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7qpd\" (UniqueName: \"kubernetes.io/projected/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-kube-api-access-x7qpd\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.542915 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.542884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-model-cache\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543108 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.542922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-dshm\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543108 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.542950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tmp-dir\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543108 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.543067    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7qpd\" (UniqueName: \"kubernetes.io/projected/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-kube-api-access-x7qpd\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543270 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.543121    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tls-certs\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543270 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.543162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-home\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543372 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.543304    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tmp-dir\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543421 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.543368    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-model-cache\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.543565 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.543547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-home\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.545477 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.545456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-dshm\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.545736 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.545720    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tls-certs\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.553210 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.553186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7qpd\" (UniqueName: \"kubernetes.io/projected/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-kube-api-access-x7qpd\") pod \"test-model-6b8164-kserve-6b8bf95675-k5xpm\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") " pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.648237 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.648198    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:05.825238 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:05.825205    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"]
Jul 22 13:15:05.826861 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:15:05.826815    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc342eeae_c30b_4e9c_b0c3_80d46c2671c4.slice/crio-c34e9e29b78302c815a5cb93978a8bd9eedd6ce7c3a857f96b863fbf246991ed WatchSource:0}: Error finding container c34e9e29b78302c815a5cb93978a8bd9eedd6ce7c3a857f96b863fbf246991ed: Status 404 returned error can't find the container with id c34e9e29b78302c815a5cb93978a8bd9eedd6ce7c3a857f96b863fbf246991ed
Jul 22 13:15:06.107050 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:06.106968    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:15:06.107480 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:06.107243    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:15:06.108028 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:06.108009    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:15:06.130026 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:06.129997    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm" event={"ID":"c342eeae-c30b-4e9c-b0c3-80d46c2671c4","Type":"ContainerStarted","Data":"b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6"}
Jul 22 13:15:06.130026 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:06.130029    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm" event={"ID":"c342eeae-c30b-4e9c-b0c3-80d46c2671c4","Type":"ContainerStarted","Data":"c34e9e29b78302c815a5cb93978a8bd9eedd6ce7c3a857f96b863fbf246991ed"}
Jul 22 13:15:06.170329 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:06.170262    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm" podStartSLOduration=1.170241235 podStartE2EDuration="1.170241235s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:15:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:15:06.169340797 +0000 UTC m=+2290.634238846" watchObservedRunningTime="2026-07-22 13:15:06.170241235 +0000 UTC m=+2290.635139285"
Jul 22 13:15:09.930772 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:09.930707    2563 patch_prober.go:28] interesting pod/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:50754->10.132.0.56:15021: read: connection reset by peer" start-of-body=
Jul 22 13:15:09.931167 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:09.930814    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:50754->10.132.0.56:15021: read: connection reset by peer"
Jul 22 13:15:10.151591 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:10.151504    2563 generic.go:356] "Generic (PLEG): container finished" podID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerID="9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5" exitCode=0
Jul 22 13:15:10.151591 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:10.151559    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" event={"ID":"d3a0b84e-b31e-4b0e-a840-a7744c784c21","Type":"ContainerDied","Data":"9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5"}
Jul 22 13:15:10.152046 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:10.152027    2563 scope.go:117] "RemoveContainer" containerID="9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5"
Jul 22 13:15:10.153139 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:10.153107    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:15:10.153235 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:10.153180    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:15:10.153235 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:10.153210    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:15:11.162172 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:11.162122    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" event={"ID":"d3a0b84e-b31e-4b0e-a840-a7744c784c21","Type":"ContainerStarted","Data":"6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297"}
Jul 22 13:15:11.481438 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:11.481397    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:15:11.481640 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:11.481618    2563 patch_prober.go:28] interesting pod/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 22 13:15:11.481697 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:11.481679    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 22 13:15:12.482518 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:12.482478    2563 patch_prober.go:28] interesting pod/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 22 13:15:12.482959 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:12.482546    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 22 13:15:13.481631 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:13.481590    2563 patch_prober.go:28] interesting pod/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 22 13:15:13.481858 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:13.481649    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 22 13:15:14.481499 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:14.481464    2563 patch_prober.go:28] interesting pod/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body=
Jul 22 13:15:14.481935 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:14.481520    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused"
Jul 22 13:15:15.485375 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:15.485342    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:15:15.485770 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:15.485605    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:15:15.486304 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:15.486288    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:15:15.649319 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:15.649287    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:15.649494 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:15.649343    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:15.665206 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:15.665173    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:16.204195 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:16.204165    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:15:28.726338 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.726301    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"]
Jul 22 13:15:28.729122 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.729104    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.734099 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.734070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c5ea07\"/\"test-model-c5ea07-kserve-self-signed-certs\""
Jul 22 13:15:28.734099 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.734088    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c5ea07\"/\"openshift-service-ca.crt\""
Jul 22 13:15:28.734270 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.734070    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-c5ea07\"/\"kube-root-ca.crt\""
Jul 22 13:15:28.734270 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.734070    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-c5ea07\"/\"default-dockercfg-khhvq\""
Jul 22 13:15:28.740159 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.740136    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"]
Jul 22 13:15:28.869221 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.869190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-tmp-dir\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.869400 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.869243    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-dshm\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.869400 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.869270    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4485fee5-05c4-4e39-b2eb-40a569c5813c-tls-certs\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.869400 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.869293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-home\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.869400 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.869310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-model-cache\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.869400 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.869329    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t6rn\" (UniqueName: \"kubernetes.io/projected/4485fee5-05c4-4e39-b2eb-40a569c5813c-kube-api-access-7t6rn\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.970602 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.970560    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-tmp-dir\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.970840 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.970630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-dshm\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.970840 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.970669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4485fee5-05c4-4e39-b2eb-40a569c5813c-tls-certs\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.970840 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.970697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-home\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.970840 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.970720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-model-cache\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.970840 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.970779    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t6rn\" (UniqueName: \"kubernetes.io/projected/4485fee5-05c4-4e39-b2eb-40a569c5813c-kube-api-access-7t6rn\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.971114 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.971046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-tmp-dir\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.971114 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.971089    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-home\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.971206 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.971185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-model-cache\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.973051 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.973032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-dshm\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.973183 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.973166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4485fee5-05c4-4e39-b2eb-40a569c5813c-tls-certs\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:28.979811 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:28.979723    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t6rn\" (UniqueName: \"kubernetes.io/projected/4485fee5-05c4-4e39-b2eb-40a569c5813c-kube-api-access-7t6rn\") pod \"test-model-c5ea07-kserve-7545ffb647-5wp8v\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") " pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:29.038380 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:29.038343    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:29.175041 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:29.174991    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"]
Jul 22 13:15:29.177269 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:15:29.177240    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4485fee5_05c4_4e39_b2eb_40a569c5813c.slice/crio-dccf82e7566a79deb674b710e936703f84ebccfa862da29ac444c21951483729 WatchSource:0}: Error finding container dccf82e7566a79deb674b710e936703f84ebccfa862da29ac444c21951483729: Status 404 returned error can't find the container with id dccf82e7566a79deb674b710e936703f84ebccfa862da29ac444c21951483729
Jul 22 13:15:29.257100 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:29.257064    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v" event={"ID":"4485fee5-05c4-4e39-b2eb-40a569c5813c","Type":"ContainerStarted","Data":"167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb"}
Jul 22 13:15:29.257232 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:29.257111    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v" event={"ID":"4485fee5-05c4-4e39-b2eb-40a569c5813c","Type":"ContainerStarted","Data":"dccf82e7566a79deb674b710e936703f84ebccfa862da29ac444c21951483729"}
Jul 22 13:15:29.275977 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:29.275931    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v" podStartSLOduration=1.275917062 podStartE2EDuration="1.275917062s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:15:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:15:29.275599474 +0000 UTC m=+2313.740497521" watchObservedRunningTime="2026-07-22 13:15:29.275917062 +0000 UTC m=+2313.740815110"
Jul 22 13:15:39.038964 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:39.038934    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:39.039361 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:39.038995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:39.051836 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:39.051810    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:15:39.316026 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:15:39.315944    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:16:08.862854 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:08.862775    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"]
Jul 22 13:16:09.788144 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:09.788113    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"]
Jul 22 13:16:23.883197 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:23.883146    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm" podUID="c342eeae-c30b-4e9c-b0c3-80d46c2671c4" containerName="main" containerID="cri-o://b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6" gracePeriod=45
Jul 22 13:16:24.135932 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.135878    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:16:24.198683 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198651    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-model-cache\") pod \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") "
Jul 22 13:16:24.198888 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tmp-dir\") pod \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") "
Jul 22 13:16:24.198888 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198782    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7qpd\" (UniqueName: \"kubernetes.io/projected/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-kube-api-access-x7qpd\") pod \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") "
Jul 22 13:16:24.198888 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198837    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-dshm\") pod \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") "
Jul 22 13:16:24.198888 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198868    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tls-certs\") pod \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") "
Jul 22 13:16:24.199123 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198895    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-home\") pod \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\" (UID: \"c342eeae-c30b-4e9c-b0c3-80d46c2671c4\") "
Jul 22 13:16:24.199123 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198934    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-model-cache" (OuterVolumeSpecName: "model-cache") pod "c342eeae-c30b-4e9c-b0c3-80d46c2671c4" (UID: "c342eeae-c30b-4e9c-b0c3-80d46c2671c4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:24.199123 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.198997    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c342eeae-c30b-4e9c-b0c3-80d46c2671c4" (UID: "c342eeae-c30b-4e9c-b0c3-80d46c2671c4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:24.199267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.199189    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-home" (OuterVolumeSpecName: "home") pod "c342eeae-c30b-4e9c-b0c3-80d46c2671c4" (UID: "c342eeae-c30b-4e9c-b0c3-80d46c2671c4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:24.199267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.199208    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:24.199267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.199228    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:24.201231 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.201202    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c342eeae-c30b-4e9c-b0c3-80d46c2671c4" (UID: "c342eeae-c30b-4e9c-b0c3-80d46c2671c4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:16:24.201341 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.201229    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-dshm" (OuterVolumeSpecName: "dshm") pod "c342eeae-c30b-4e9c-b0c3-80d46c2671c4" (UID: "c342eeae-c30b-4e9c-b0c3-80d46c2671c4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:24.201341 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.201262    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-kube-api-access-x7qpd" (OuterVolumeSpecName: "kube-api-access-x7qpd") pod "c342eeae-c30b-4e9c-b0c3-80d46c2671c4" (UID: "c342eeae-c30b-4e9c-b0c3-80d46c2671c4"). InnerVolumeSpecName "kube-api-access-x7qpd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:16:24.300421 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.300385    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7qpd\" (UniqueName: \"kubernetes.io/projected/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-kube-api-access-x7qpd\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:24.300421 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.300414    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:24.300421 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.300427    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:24.300665 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.300439    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c342eeae-c30b-4e9c-b0c3-80d46c2671c4-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:24.519909 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.519875    2563 generic.go:356] "Generic (PLEG): container finished" podID="c342eeae-c30b-4e9c-b0c3-80d46c2671c4" containerID="b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6" exitCode=0
Jul 22 13:16:24.520083 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.519916    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"
Jul 22 13:16:24.520083 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.519956    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm" event={"ID":"c342eeae-c30b-4e9c-b0c3-80d46c2671c4","Type":"ContainerDied","Data":"b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6"}
Jul 22 13:16:24.520083 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.520005    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm" event={"ID":"c342eeae-c30b-4e9c-b0c3-80d46c2671c4","Type":"ContainerDied","Data":"c34e9e29b78302c815a5cb93978a8bd9eedd6ce7c3a857f96b863fbf246991ed"}
Jul 22 13:16:24.520083 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.520022    2563 scope.go:117] "RemoveContainer" containerID="b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6"
Jul 22 13:16:24.529427 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.529410    2563 scope.go:117] "RemoveContainer" containerID="b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6"
Jul 22 13:16:24.529673 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:16:24.529655    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6\": container with ID starting with b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6 not found: ID does not exist" containerID="b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6"
Jul 22 13:16:24.529741 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.529681    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6"} err="failed to get container status \"b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6\": rpc error: code = NotFound desc = could not find container \"b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6\": container with ID starting with b11f9a6b9f438554833a94ea9f6dab4147a464b0d0c37b9cefa4277bff3b88e6 not found: ID does not exist"
Jul 22 13:16:24.540984 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.540960    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"]
Jul 22 13:16:24.545480 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.545460    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-6b8164/test-model-6b8164-kserve-6b8bf95675-k5xpm"]
Jul 22 13:16:24.808971 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:24.808860    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v" podUID="4485fee5-05c4-4e39-b2eb-40a569c5813c" containerName="main" containerID="cri-o://167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb" gracePeriod=45
Jul 22 13:16:25.067203 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.067145    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:16:25.108135 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108110    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4485fee5-05c4-4e39-b2eb-40a569c5813c-tls-certs\") pod \"4485fee5-05c4-4e39-b2eb-40a569c5813c\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") "
Jul 22 13:16:25.108272 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108175    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-dshm\") pod \"4485fee5-05c4-4e39-b2eb-40a569c5813c\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") "
Jul 22 13:16:25.108319 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108289    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7t6rn\" (UniqueName: \"kubernetes.io/projected/4485fee5-05c4-4e39-b2eb-40a569c5813c-kube-api-access-7t6rn\") pod \"4485fee5-05c4-4e39-b2eb-40a569c5813c\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") "
Jul 22 13:16:25.108352 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108324    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-home\") pod \"4485fee5-05c4-4e39-b2eb-40a569c5813c\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") "
Jul 22 13:16:25.108397 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108354    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-tmp-dir\") pod \"4485fee5-05c4-4e39-b2eb-40a569c5813c\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") "
Jul 22 13:16:25.108461 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-model-cache\") pod \"4485fee5-05c4-4e39-b2eb-40a569c5813c\" (UID: \"4485fee5-05c4-4e39-b2eb-40a569c5813c\") "
Jul 22 13:16:25.108734 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108678    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-home" (OuterVolumeSpecName: "home") pod "4485fee5-05c4-4e39-b2eb-40a569c5813c" (UID: "4485fee5-05c4-4e39-b2eb-40a569c5813c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:25.108925 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108691    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4485fee5-05c4-4e39-b2eb-40a569c5813c" (UID: "4485fee5-05c4-4e39-b2eb-40a569c5813c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:25.108925 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.108779    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-model-cache" (OuterVolumeSpecName: "model-cache") pod "4485fee5-05c4-4e39-b2eb-40a569c5813c" (UID: "4485fee5-05c4-4e39-b2eb-40a569c5813c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:25.110532 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.110513    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-dshm" (OuterVolumeSpecName: "dshm") pod "4485fee5-05c4-4e39-b2eb-40a569c5813c" (UID: "4485fee5-05c4-4e39-b2eb-40a569c5813c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:25.110946 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.110929    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4485fee5-05c4-4e39-b2eb-40a569c5813c-kube-api-access-7t6rn" (OuterVolumeSpecName: "kube-api-access-7t6rn") pod "4485fee5-05c4-4e39-b2eb-40a569c5813c" (UID: "4485fee5-05c4-4e39-b2eb-40a569c5813c"). InnerVolumeSpecName "kube-api-access-7t6rn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:16:25.111011 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.110940    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4485fee5-05c4-4e39-b2eb-40a569c5813c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4485fee5-05c4-4e39-b2eb-40a569c5813c" (UID: "4485fee5-05c4-4e39-b2eb-40a569c5813c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:16:25.210267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.210235    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:25.210267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.210261    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4485fee5-05c4-4e39-b2eb-40a569c5813c-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:25.210267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.210270    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:25.210489 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.210279    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7t6rn\" (UniqueName: \"kubernetes.io/projected/4485fee5-05c4-4e39-b2eb-40a569c5813c-kube-api-access-7t6rn\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:25.210489 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.210288    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:25.210489 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.210296    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4485fee5-05c4-4e39-b2eb-40a569c5813c-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:25.526578 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.526545    2563 generic.go:356] "Generic (PLEG): container finished" podID="4485fee5-05c4-4e39-b2eb-40a569c5813c" containerID="167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb" exitCode=0
Jul 22 13:16:25.526777 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.526588    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"
Jul 22 13:16:25.526777 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.526628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v" event={"ID":"4485fee5-05c4-4e39-b2eb-40a569c5813c","Type":"ContainerDied","Data":"167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb"}
Jul 22 13:16:25.526777 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.526669    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v" event={"ID":"4485fee5-05c4-4e39-b2eb-40a569c5813c","Type":"ContainerDied","Data":"dccf82e7566a79deb674b710e936703f84ebccfa862da29ac444c21951483729"}
Jul 22 13:16:25.526777 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.526690    2563 scope.go:117] "RemoveContainer" containerID="167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb"
Jul 22 13:16:25.535680 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.535662    2563 scope.go:117] "RemoveContainer" containerID="167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb"
Jul 22 13:16:25.535999 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:16:25.535980    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb\": container with ID starting with 167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb not found: ID does not exist" containerID="167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb"
Jul 22 13:16:25.536055 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.536007    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb"} err="failed to get container status \"167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb\": rpc error: code = NotFound desc = could not find container \"167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb\": container with ID starting with 167f316f9bf08d1f9df77e5ba6db46d60089ed0ccd07206dad9a6e2634c5bfbb not found: ID does not exist"
Jul 22 13:16:25.547450 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.547424    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"]
Jul 22 13:16:25.551030 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:25.551010    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-c5ea07/test-model-c5ea07-kserve-7545ffb647-5wp8v"]
Jul 22 13:16:26.020514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:26.020484    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4485fee5-05c4-4e39-b2eb-40a569c5813c" path="/var/lib/kubelet/pods/4485fee5-05c4-4e39-b2eb-40a569c5813c/volumes"
Jul 22 13:16:26.020887 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:26.020874    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c342eeae-c30b-4e9c-b0c3-80d46c2671c4" path="/var/lib/kubelet/pods/c342eeae-c30b-4e9c-b0c3-80d46c2671c4/volumes"
Jul 22 13:16:31.547380 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:31.547345    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"]
Jul 22 13:16:31.547816 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:31.547555    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy" containerID="cri-o://7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9" gracePeriod=30
Jul 22 13:16:36.109532 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.109488    2563 patch_prober.go:28] interesting pod/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 13:16:36.109924 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.109551    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:16:36.800677 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.800642    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:16:36.923637 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923553    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-credential-socket\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.923637 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923602    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rgxhg\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-kube-api-access-rgxhg\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.923881 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923637    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-podinfo\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.923881 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923663    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-socket\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.923881 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923695    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-certs\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.923881 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923727    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-data\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.923881 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf478d0f-0a5a-498e-9589-e2b4ead49452-istiod-ca-cert\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.923881 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923865    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-token\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.924184 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923910    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:36.924184 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.923924    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-envoy\") pod \"cf478d0f-0a5a-498e-9589-e2b4ead49452\" (UID: \"cf478d0f-0a5a-498e-9589-e2b4ead49452\") "
Jul 22 13:16:36.924184 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924100    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:36.924184 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924114    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:36.924394 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924277    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf478d0f-0a5a-498e-9589-e2b4ead49452-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:16:36.924394 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924385    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf478d0f-0a5a-498e-9589-e2b4ead49452-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:36.924501 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924405    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:36.924611 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924561    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:36.924671 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924614    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:36.924827 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.924802    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-data" (OuterVolumeSpecName: "istio-data") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:36.926108 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.926086    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-kube-api-access-rgxhg" (OuterVolumeSpecName: "kube-api-access-rgxhg") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "kube-api-access-rgxhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:16:36.926199 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.926103    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:36.926657 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.926637    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-token" (OuterVolumeSpecName: "istio-token") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:16:36.926766 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:36.926654    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cf478d0f-0a5a-498e-9589-e2b4ead49452" (UID: "cf478d0f-0a5a-498e-9589-e2b4ead49452"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:16:37.026024 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.025985    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rgxhg\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-kube-api-access-rgxhg\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:37.026024 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.026016    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:37.026024 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.026025    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:37.026024 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.026034    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:37.026291 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.026041    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf478d0f-0a5a-498e-9589-e2b4ead49452-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:37.588373 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.588339    2563 generic.go:356] "Generic (PLEG): container finished" podID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerID="7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9" exitCode=0
Jul 22 13:16:37.588856 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.588385    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" event={"ID":"cf478d0f-0a5a-498e-9589-e2b4ead49452","Type":"ContainerDied","Data":"7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9"}
Jul 22 13:16:37.588856 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.588435    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj" event={"ID":"cf478d0f-0a5a-498e-9589-e2b4ead49452","Type":"ContainerDied","Data":"a727c5f286c13bbc9260719d423b213c99d42fddad0f0132d23828ccc46fca75"}
Jul 22 13:16:37.588856 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.588389    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"
Jul 22 13:16:37.588856 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.588464    2563 scope.go:117] "RemoveContainer" containerID="7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9"
Jul 22 13:16:37.598270 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.598253    2563 scope.go:117] "RemoveContainer" containerID="4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36"
Jul 22 13:16:37.606484 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.606468    2563 scope.go:117] "RemoveContainer" containerID="7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9"
Jul 22 13:16:37.606806 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:16:37.606779    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9\": container with ID starting with 7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9 not found: ID does not exist" containerID="7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9"
Jul 22 13:16:37.606875 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.606818    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9"} err="failed to get container status \"7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9\": rpc error: code = NotFound desc = could not find container \"7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9\": container with ID starting with 7e62885089cafa8acfd8d3408cd6d02e41ec3fccc18a7dba2ae9ce07586c58a9 not found: ID does not exist"
Jul 22 13:16:37.606875 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.606843    2563 scope.go:117] "RemoveContainer" containerID="4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36"
Jul 22 13:16:37.607151 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:16:37.607123    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36\": container with ID starting with 4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36 not found: ID does not exist" containerID="4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36"
Jul 22 13:16:37.607151 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.607159    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36"} err="failed to get container status \"4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36\": rpc error: code = NotFound desc = could not find container \"4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36\": container with ID starting with 4cb496d1cadb78d2da0c2df943026ea5f1f75acdbff8a67ae495a7aaa9796d36 not found: ID does not exist"
Jul 22 13:16:37.612342 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.612322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"]
Jul 22 13:16:37.616136 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:37.616115    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-6b8164-openshift-default-67c55f567f-xt4cj"]
Jul 22 13:16:38.018669 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:38.018637    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" path="/var/lib/kubelet/pods/cf478d0f-0a5a-498e-9589-e2b4ead49452/volumes"
Jul 22 13:16:53.592600 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:53.592568    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"]
Jul 22 13:16:53.593164 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:53.592790    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy" containerID="cri-o://6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297" gracePeriod=30
Jul 22 13:16:54.457803 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.457764    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"]
Jul 22 13:16:54.458322 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458291    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c342eeae-c30b-4e9c-b0c3-80d46c2671c4" containerName="main"
Jul 22 13:16:54.458322 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458322    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c342eeae-c30b-4e9c-b0c3-80d46c2671c4" containerName="main"
Jul 22 13:16:54.458514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458353    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy"
Jul 22 13:16:54.458514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458361    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy"
Jul 22 13:16:54.458514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458389    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy"
Jul 22 13:16:54.458514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458397    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy"
Jul 22 13:16:54.458514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458407    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4485fee5-05c4-4e39-b2eb-40a569c5813c" containerName="main"
Jul 22 13:16:54.458514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458414    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4485fee5-05c4-4e39-b2eb-40a569c5813c" containerName="main"
Jul 22 13:16:54.459262 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458519    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy"
Jul 22 13:16:54.459262 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458529    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4485fee5-05c4-4e39-b2eb-40a569c5813c" containerName="main"
Jul 22 13:16:54.459262 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.458542    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c342eeae-c30b-4e9c-b0c3-80d46c2671c4" containerName="main"
Jul 22 13:16:54.463204 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.463184    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.465818 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.465795    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-266bd1-openshift-default-dockercfg-kxgd9\""
Jul 22 13:16:54.472825 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.472804    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"]
Jul 22 13:16:54.487758 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.487714    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1af07927-17d6-4532-afa8-76251ea4b169-istio-podinfo\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.487858 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.487787    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-certs\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.487858 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.487841    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-socket\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.487934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.487868    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-envoy\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.487934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.487892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfrqk\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-kube-api-access-mfrqk\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.487934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.487915    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-istio-token\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.487934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.487929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1af07927-17d6-4532-afa8-76251ea4b169-istiod-ca-cert\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.488079 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.488002    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-data\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.488079 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.488032    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-credential-socket\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.588890 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.588851    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-socket\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589081 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.588897    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-envoy\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589081 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.588928    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mfrqk\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-kube-api-access-mfrqk\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589081 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.588963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-istio-token\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589081 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.588985    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1af07927-17d6-4532-afa8-76251ea4b169-istiod-ca-cert\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589081 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589032    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-data\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589081 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-credential-socket\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589418 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589138    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1af07927-17d6-4532-afa8-76251ea4b169-istio-podinfo\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589418 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589187    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-certs\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589418 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589370    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-socket\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589590 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589548    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-certs\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589590 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589570    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-data\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589680 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589664    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-credential-socket\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.589819 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.589801    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1af07927-17d6-4532-afa8-76251ea4b169-istiod-ca-cert\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.591379 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.591355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-envoy\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.591828 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.591808    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1af07927-17d6-4532-afa8-76251ea4b169-istio-podinfo\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.597684 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.597662    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-istio-token\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.597996 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.597686    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfrqk\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-kube-api-access-mfrqk\") pod \"e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") " pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.773779 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.773679    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:54.914496 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.914335    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"]
Jul 22 13:16:54.917089 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:16:54.917055    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af07927_17d6_4532_afa8_76251ea4b169.slice/crio-eb7c0c561726dddff8e9feb217f5995d34a8fcd5abca590bfa9bbbb63ce92d9d WatchSource:0}: Error finding container eb7c0c561726dddff8e9feb217f5995d34a8fcd5abca590bfa9bbbb63ce92d9d: Status 404 returned error can't find the container with id eb7c0c561726dddff8e9feb217f5995d34a8fcd5abca590bfa9bbbb63ce92d9d
Jul 22 13:16:54.919301 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.919272    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:16:54.919383 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.919338    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:16:54.919383 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:54.919367    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:16:55.677304 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:55.677261    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" event={"ID":"1af07927-17d6-4532-afa8-76251ea4b169","Type":"ContainerStarted","Data":"2f40fbdac6c5d8597f6e28ca7cf81c2e81c1f2f36df7b883e000e701428983b2"}
Jul 22 13:16:55.677675 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:55.677335    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" event={"ID":"1af07927-17d6-4532-afa8-76251ea4b169","Type":"ContainerStarted","Data":"eb7c0c561726dddff8e9feb217f5995d34a8fcd5abca590bfa9bbbb63ce92d9d"}
Jul 22 13:16:55.698550 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:55.698507    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" podStartSLOduration=1.6984932339999999 podStartE2EDuration="1.698493234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:16:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:16:55.696458574 +0000 UTC m=+2400.161356651" watchObservedRunningTime="2026-07-22 13:16:55.698493234 +0000 UTC m=+2400.163391282"
Jul 22 13:16:55.774800 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:55.774770    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:55.779396 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:55.779375    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:56.683608 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:56.683524    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:56.684579 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:56.684559    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:16:58.847849 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.847824    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:16:58.923247 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923166    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istiod-ca-cert\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923247 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923203    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-credential-socket\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923247 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923223    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-token\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923512 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923264    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-podinfo\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923512 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923334    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-socket\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923512 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923372    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mv9s\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-kube-api-access-9mv9s\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923512 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923423    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-data\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923512 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923454    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-envoy\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923512 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923478    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:58.923512 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923511    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-certs\") pod \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\" (UID: \"d3a0b84e-b31e-4b0e-a840-a7744c784c21\") "
Jul 22 13:16:58.923918 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923699    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:58.923918 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923728    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:58.924026 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923912    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:16:58.924026 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923909    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:58.924026 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923945    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:58.924026 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.923961    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:58.924792 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.924765    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-data" (OuterVolumeSpecName: "istio-data") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:58.925849 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.925828    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-kube-api-access-9mv9s" (OuterVolumeSpecName: "kube-api-access-9mv9s") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "kube-api-access-9mv9s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:16:58.926074 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.926059    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:16:58.926236 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.926215    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-token" (OuterVolumeSpecName: "istio-token") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:16:58.926339 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:58.926231    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d3a0b84e-b31e-4b0e-a840-a7744c784c21" (UID: "d3a0b84e-b31e-4b0e-a840-a7744c784c21"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:16:59.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.024460    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:59.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.024498    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:59.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.024510    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:59.024501 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.024522    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:59.024815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.024536    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d3a0b84e-b31e-4b0e-a840-a7744c784c21-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:59.024815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.024554    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mv9s\" (UniqueName: \"kubernetes.io/projected/d3a0b84e-b31e-4b0e-a840-a7744c784c21-kube-api-access-9mv9s\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:16:59.700763 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.700721    2563 generic.go:356] "Generic (PLEG): container finished" podID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerID="6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297" exitCode=0
Jul 22 13:16:59.700957 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.700777    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" event={"ID":"d3a0b84e-b31e-4b0e-a840-a7744c784c21","Type":"ContainerDied","Data":"6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297"}
Jul 22 13:16:59.700957 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.700800    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"
Jul 22 13:16:59.700957 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.700834    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj" event={"ID":"d3a0b84e-b31e-4b0e-a840-a7744c784c21","Type":"ContainerDied","Data":"5f5a3e97395923b12ff691c090d9383155c7d971aa846ce12e2c20a2ae1fe064"}
Jul 22 13:16:59.700957 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.700856    2563 scope.go:117] "RemoveContainer" containerID="6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297"
Jul 22 13:16:59.710384 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.710216    2563 scope.go:117] "RemoveContainer" containerID="9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5"
Jul 22 13:16:59.718163 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.718137    2563 scope.go:117] "RemoveContainer" containerID="6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297"
Jul 22 13:16:59.718429 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:16:59.718411    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297\": container with ID starting with 6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297 not found: ID does not exist" containerID="6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297"
Jul 22 13:16:59.718492 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.718437    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297"} err="failed to get container status \"6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297\": rpc error: code = NotFound desc = could not find container \"6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297\": container with ID starting with 6e539a93171bc14e2a6aec3e9aaba37d5817591039a63f10ba80033c96147297 not found: ID does not exist"
Jul 22 13:16:59.718492 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.718454    2563 scope.go:117] "RemoveContainer" containerID="9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5"
Jul 22 13:16:59.718677 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:16:59.718662    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5\": container with ID starting with 9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5 not found: ID does not exist" containerID="9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5"
Jul 22 13:16:59.718723 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.718679    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5"} err="failed to get container status \"9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5\": rpc error: code = NotFound desc = could not find container \"9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5\": container with ID starting with 9c1374843c8abd04b47e5a093804570f57467265851c0eab7d8ea9fefaf937b5 not found: ID does not exist"
Jul 22 13:16:59.723404 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.723379    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"]
Jul 22 13:16:59.727367 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:16:59.727347    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-c5ea07-openshift-default-5975f77bcc-rbwqj"]
Jul 22 13:17:00.017867 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:00.017793    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" path="/var/lib/kubelet/pods/d3a0b84e-b31e-4b0e-a840-a7744c784c21/volumes"
Jul 22 13:17:02.910966 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.910924    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"]
Jul 22 13:17:02.911458 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.911322    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy"
Jul 22 13:17:02.911458 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.911332    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy"
Jul 22 13:17:02.911458 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.911340    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy"
Jul 22 13:17:02.911458 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.911345    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy"
Jul 22 13:17:02.911458 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.911405    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy"
Jul 22 13:17:02.911458 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.911414    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf478d0f-0a5a-498e-9589-e2b4ead49452" containerName="istio-proxy"
Jul 22 13:17:02.914271 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.914252    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.917068 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.917051    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-0d04d7-openshift-default-dockercfg-5bcjf\""
Jul 22 13:17:02.926129 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.926102    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"]
Jul 22 13:17:02.956341 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956315    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-credential-socket\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956496 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956367    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-certs\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956496 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956431    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/083bc315-a956-4431-a053-79f5d8293ad3-istio-podinfo\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956496 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956464    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-istio-token\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956688 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-data\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956688 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956634    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jv79p\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-kube-api-access-jv79p\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956825 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956685    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-socket\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956825 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956724    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/083bc315-a956-4431-a053-79f5d8293ad3-istiod-ca-cert\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:02.956825 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:02.956763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-envoy\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057600 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057565    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-socket\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057612    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/083bc315-a956-4431-a053-79f5d8293ad3-istiod-ca-cert\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057630    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-envoy\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-credential-socket\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-certs\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057703    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/083bc315-a956-4431-a053-79f5d8293ad3-istio-podinfo\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057717    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-istio-token\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-data\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.057815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.057801    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jv79p\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-kube-api-access-jv79p\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.058248 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.058105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-socket\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.058248 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.058210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-credential-socket\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.058370 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.058284    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-data\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.058492 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.058472    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/083bc315-a956-4431-a053-79f5d8293ad3-istiod-ca-cert\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.058553 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.058531    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-certs\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.060179 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.060160    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-envoy\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.060510 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.060488    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/083bc315-a956-4431-a053-79f5d8293ad3-istio-podinfo\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.066090 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.066068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-istio-token\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.066677 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.066657    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jv79p\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-kube-api-access-jv79p\") pod \"e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") " pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.224970 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.224940    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:03.364670 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.364644    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"]
Jul 22 13:17:03.366394 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:17:03.366366    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod083bc315_a956_4431_a053_79f5d8293ad3.slice/crio-f8573e6abd1066a042b88fb5b168b1b9c1ef9234eff965533b1c88b4955208a2 WatchSource:0}: Error finding container f8573e6abd1066a042b88fb5b168b1b9c1ef9234eff965533b1c88b4955208a2: Status 404 returned error can't find the container with id f8573e6abd1066a042b88fb5b168b1b9c1ef9234eff965533b1c88b4955208a2
Jul 22 13:17:03.368535 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.368505    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:03.368619 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.368574    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:03.368619 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.368603    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:03.721581 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.721545    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" event={"ID":"083bc315-a956-4431-a053-79f5d8293ad3","Type":"ContainerStarted","Data":"4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b"}
Jul 22 13:17:03.721772 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.721599    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" event={"ID":"083bc315-a956-4431-a053-79f5d8293ad3","Type":"ContainerStarted","Data":"f8573e6abd1066a042b88fb5b168b1b9c1ef9234eff965533b1c88b4955208a2"}
Jul 22 13:17:03.742718 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:03.742673    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podStartSLOduration=1.7426568009999999 podStartE2EDuration="1.742656801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:17:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:17:03.740987277 +0000 UTC m=+2408.205885353" watchObservedRunningTime="2026-07-22 13:17:03.742656801 +0000 UTC m=+2408.207554850"
Jul 22 13:17:04.225823 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:04.225790    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:04.230801 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:04.230776    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:04.725957 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:04.725930    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:04.727010 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:04.726995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:17.797076 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:17.797042    2563 generic.go:356] "Generic (PLEG): container finished" podID="1af07927-17d6-4532-afa8-76251ea4b169" containerID="2f40fbdac6c5d8597f6e28ca7cf81c2e81c1f2f36df7b883e000e701428983b2" exitCode=0
Jul 22 13:17:17.797514 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:17.797115    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" event={"ID":"1af07927-17d6-4532-afa8-76251ea4b169","Type":"ContainerDied","Data":"2f40fbdac6c5d8597f6e28ca7cf81c2e81c1f2f36df7b883e000e701428983b2"}
Jul 22 13:17:17.797560 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:17.797518    2563 scope.go:117] "RemoveContainer" containerID="2f40fbdac6c5d8597f6e28ca7cf81c2e81c1f2f36df7b883e000e701428983b2"
Jul 22 13:17:17.798471 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:17.798447    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:17.798527 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:17.798513    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:17.798562 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:17.798543    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:18.804292 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:18.804256    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" event={"ID":"1af07927-17d6-4532-afa8-76251ea4b169","Type":"ContainerStarted","Data":"e7984997de28ab742ed6e4ca9d46012e7eda3a1d6048a74809b236c1c2466c98"}
Jul 22 13:17:19.774775 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:19.773989    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:17:19.774775 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:19.774411    2563 patch_prober.go:28] interesting pod/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 22 13:17:19.774775 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:19.774466    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 22 13:17:20.768323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:20.767943    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 22 13:17:20.768323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:20.768031    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 22 13:17:20.774465 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:20.774361    2563 patch_prober.go:28] interesting pod/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 22 13:17:20.774465 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:20.774411    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 22 13:17:21.774604 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.774564    2563 patch_prober.go:28] interesting pod/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body=
Jul 22 13:17:21.775125 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.774638    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused"
Jul 22 13:17:21.818822 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.818791    2563 generic.go:356] "Generic (PLEG): container finished" podID="083bc315-a956-4431-a053-79f5d8293ad3" containerID="4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b" exitCode=0
Jul 22 13:17:21.819007 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.818837    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" event={"ID":"083bc315-a956-4431-a053-79f5d8293ad3","Type":"ContainerDied","Data":"4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b"}
Jul 22 13:17:21.819240 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.819227    2563 scope.go:117] "RemoveContainer" containerID="4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b"
Jul 22 13:17:21.820295 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.820268    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:21.820368 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.820354    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:21.820404 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:21.820392    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:17:22.778625 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:22.778591    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:17:22.779089 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:22.778936    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:17:22.779603 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:22.779588    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:17:22.825694 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:22.825659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" event={"ID":"083bc315-a956-4431-a053-79f5d8293ad3","Type":"ContainerStarted","Data":"3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56"}
Jul 22 13:17:23.225528 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:23.225496    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:23.225700 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:23.225619    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:17:23.225700 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:23.225681    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 22 13:17:24.225796 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:24.225737    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:17:24.226208 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:24.225815    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 22 13:17:25.225807 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:25.225764    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:17:25.226340 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:25.225825    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 22 13:17:26.225574 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:26.225538    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:17:26.225778 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:26.225592    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 22 13:17:27.225533 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:27.225484    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:17:27.225982 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:27.225551    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 22 13:17:28.225315 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:28.225279    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:17:28.225496 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:28.225338    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 22 13:17:29.225403 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:29.225367    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Jul 22 13:17:29.225872 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:29.225431    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Jul 22 13:17:30.229759 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:30.229713    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:30.230219 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:30.230203    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:30.231016 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:30.230990    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:17:44.844965 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:44.844928    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"]
Jul 22 13:17:44.845821 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:44.845733    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy" containerID="cri-o://e7984997de28ab742ed6e4ca9d46012e7eda3a1d6048a74809b236c1c2466c98" gracePeriod=30
Jul 22 13:17:49.964720 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:49.964680    2563 generic.go:356] "Generic (PLEG): container finished" podID="1af07927-17d6-4532-afa8-76251ea4b169" containerID="e7984997de28ab742ed6e4ca9d46012e7eda3a1d6048a74809b236c1c2466c98" exitCode=0
Jul 22 13:17:49.965110 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:49.964772    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" event={"ID":"1af07927-17d6-4532-afa8-76251ea4b169","Type":"ContainerDied","Data":"e7984997de28ab742ed6e4ca9d46012e7eda3a1d6048a74809b236c1c2466c98"}
Jul 22 13:17:49.965110 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:49.964830    2563 scope.go:117] "RemoveContainer" containerID="2f40fbdac6c5d8597f6e28ca7cf81c2e81c1f2f36df7b883e000e701428983b2"
Jul 22 13:17:50.113817 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.113761    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:17:50.208430 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208391    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-certs\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208430 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208438    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-istio-token\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208469    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-data\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208504    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-socket\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208533    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mfrqk\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-kube-api-access-mfrqk\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208556    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-envoy\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208580    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1af07927-17d6-4532-afa8-76251ea4b169-istiod-ca-cert\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208623    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1af07927-17d6-4532-afa8-76251ea4b169-istio-podinfo\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208653    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-credential-socket\") pod \"1af07927-17d6-4532-afa8-76251ea4b169\" (UID: \"1af07927-17d6-4532-afa8-76251ea4b169\") "
Jul 22 13:17:50.208701 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.208690    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:17:50.209173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.209067    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.209173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.209088    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af07927-17d6-4532-afa8-76251ea4b169-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:17:50.209286 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.209263    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:17:50.209516 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.209499    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:17:50.210051 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.210023    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-data" (OuterVolumeSpecName: "istio-data") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:17:50.210935 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.210908    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-kube-api-access-mfrqk" (OuterVolumeSpecName: "kube-api-access-mfrqk") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "kube-api-access-mfrqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:17:50.211035 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.210915    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-istio-token" (OuterVolumeSpecName: "istio-token") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:17:50.211375 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.211357    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1af07927-17d6-4532-afa8-76251ea4b169-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:17:50.211375 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.211357    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1af07927-17d6-4532-afa8-76251ea4b169" (UID: "1af07927-17d6-4532-afa8-76251ea4b169"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:17:50.310539 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310511    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.310539 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310535    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.310539 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310544    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.310772 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310554    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mfrqk\" (UniqueName: \"kubernetes.io/projected/1af07927-17d6-4532-afa8-76251ea4b169-kube-api-access-mfrqk\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.310772 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310563    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.310772 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310571    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1af07927-17d6-4532-afa8-76251ea4b169-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.310772 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310579    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1af07927-17d6-4532-afa8-76251ea4b169-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.310772 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.310586    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1af07927-17d6-4532-afa8-76251ea4b169-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:17:50.971989 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.971950    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct" event={"ID":"1af07927-17d6-4532-afa8-76251ea4b169","Type":"ContainerDied","Data":"eb7c0c561726dddff8e9feb217f5995d34a8fcd5abca590bfa9bbbb63ce92d9d"}
Jul 22 13:17:50.971989 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.971985    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"
Jul 22 13:17:50.972499 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.972041    2563 scope.go:117] "RemoveContainer" containerID="e7984997de28ab742ed6e4ca9d46012e7eda3a1d6048a74809b236c1c2466c98"
Jul 22 13:17:50.994476 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:50.994449    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"]
Jul 22 13:17:51.000214 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:51.000192    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-266bd1-openshift-default-76f96bc4c7-zmdct"]
Jul 22 13:17:52.019001 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:17:52.018968    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1af07927-17d6-4532-afa8-76251ea4b169" path="/var/lib/kubelet/pods/1af07927-17d6-4532-afa8-76251ea4b169/volumes"
Jul 22 13:18:11.740478 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:11.740446    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"]
Jul 22 13:18:11.740971 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:11.740679    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" containerID="cri-o://3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56" gracePeriod=30
Jul 22 13:18:12.664887 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.664855    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"]
Jul 22 13:18:12.665278 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.665265    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy"
Jul 22 13:18:12.665322 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.665281    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy"
Jul 22 13:18:12.665322 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.665305    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy"
Jul 22 13:18:12.665322 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.665312    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy"
Jul 22 13:18:12.665420 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.665371    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy"
Jul 22 13:18:12.665420 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.665384    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3a0b84e-b31e-4b0e-a840-a7744c784c21" containerName="istio-proxy"
Jul 22 13:18:12.665420 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.665390    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af07927-17d6-4532-afa8-76251ea4b169" containerName="istio-proxy"
Jul 22 13:18:12.668408 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.668391    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.671479 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.671454    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-634440-openshift-default-dockercfg-jhb9h\""
Jul 22 13:18:12.681310 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.681289    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"]
Jul 22 13:18:12.814945 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.814904    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-istio-token\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.814945 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.814947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-socket\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.815449 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.814980    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05794723-c5b4-41e2-bc62-191dfa59771f-istiod-ca-cert\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.815449 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.815027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-envoy\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.815449 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.815062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05794723-c5b4-41e2-bc62-191dfa59771f-istio-podinfo\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.815449 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.815088    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8nk5\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-kube-api-access-k8nk5\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.815449 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.815123    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-data\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.815449 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.815202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-credential-socket\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.815449 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.815242    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-certs\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916652 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916553    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-istio-token\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916652 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916598    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-socket\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916652 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05794723-c5b4-41e2-bc62-191dfa59771f-istiod-ca-cert\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916982 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-envoy\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916982 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916686    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05794723-c5b4-41e2-bc62-191dfa59771f-istio-podinfo\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916982 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916707    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8nk5\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-kube-api-access-k8nk5\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916982 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916739    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-data\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916982 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916802    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-credential-socket\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.916982 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.916837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-certs\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.917214 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.917086    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-socket\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.917435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.917376    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-certs\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.917435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.917418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-data\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.917558 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.917533    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05794723-c5b4-41e2-bc62-191dfa59771f-istiod-ca-cert\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.917649 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.917629    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-credential-socket\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.919086 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.919061    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-envoy\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.919272 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.919254    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05794723-c5b4-41e2-bc62-191dfa59771f-istio-podinfo\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.926346 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.926318    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8nk5\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-kube-api-access-k8nk5\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.926610 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.926588    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-istio-token\") pod \"e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") " pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:12.978639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:12.978612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:13.115558 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:13.115534    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"]
Jul 22 13:18:13.118213 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:18:13.118186    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05794723_c5b4_41e2_bc62_191dfa59771f.slice/crio-b4d60e2b8a4571c347d81035ea46f14c1b596e57efdbf4df45e0aed783ca2c0e WatchSource:0}: Error finding container b4d60e2b8a4571c347d81035ea46f14c1b596e57efdbf4df45e0aed783ca2c0e: Status 404 returned error can't find the container with id b4d60e2b8a4571c347d81035ea46f14c1b596e57efdbf4df45e0aed783ca2c0e
Jul 22 13:18:13.120891 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:13.120859    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:13.121021 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:13.120933    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:13.121021 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:13.120977    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:14.082710 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:14.082677    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" event={"ID":"05794723-c5b4-41e2-bc62-191dfa59771f","Type":"ContainerStarted","Data":"f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5"}
Jul 22 13:18:14.083122 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:14.082727    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" event={"ID":"05794723-c5b4-41e2-bc62-191dfa59771f","Type":"ContainerStarted","Data":"b4d60e2b8a4571c347d81035ea46f14c1b596e57efdbf4df45e0aed783ca2c0e"}
Jul 22 13:18:14.104561 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:14.104505    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podStartSLOduration=2.104489504 podStartE2EDuration="2.104489504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:18:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:18:14.102725293 +0000 UTC m=+2478.567623341" watchObservedRunningTime="2026-07-22 13:18:14.104489504 +0000 UTC m=+2478.569387553"
Jul 22 13:18:14.979032 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:14.978992    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:14.984124 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:14.984099    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:15.087896 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:15.087860    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:15.088681 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:15.088663    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:15.232388 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:15.232303    2563 patch_prober.go:28] interesting pod/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 13:18:15.232388 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:15.232374    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:18:16.987973 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:16.987943    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:18:17.099066 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.098985    2563 generic.go:356] "Generic (PLEG): container finished" podID="083bc315-a956-4431-a053-79f5d8293ad3" containerID="3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56" exitCode=0
Jul 22 13:18:17.099066 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.099035    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"
Jul 22 13:18:17.099305 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.099075    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" event={"ID":"083bc315-a956-4431-a053-79f5d8293ad3","Type":"ContainerDied","Data":"3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56"}
Jul 22 13:18:17.099305 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.099141    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f" event={"ID":"083bc315-a956-4431-a053-79f5d8293ad3","Type":"ContainerDied","Data":"f8573e6abd1066a042b88fb5b168b1b9c1ef9234eff965533b1c88b4955208a2"}
Jul 22 13:18:17.099305 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.099170    2563 scope.go:117] "RemoveContainer" containerID="3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56"
Jul 22 13:18:17.107802 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.107785    2563 scope.go:117] "RemoveContainer" containerID="4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b"
Jul 22 13:18:17.115395 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.115379    2563 scope.go:117] "RemoveContainer" containerID="3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56"
Jul 22 13:18:17.115642 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:18:17.115624    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56\": container with ID starting with 3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56 not found: ID does not exist" containerID="3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56"
Jul 22 13:18:17.115691 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.115652    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56"} err="failed to get container status \"3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56\": rpc error: code = NotFound desc = could not find container \"3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56\": container with ID starting with 3cc33c41455d8fdea6391c25accd8f86e431cf93a527e40c3b0f809a77e3ee56 not found: ID does not exist"
Jul 22 13:18:17.115691 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.115670    2563 scope.go:117] "RemoveContainer" containerID="4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b"
Jul 22 13:18:17.115967 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:18:17.115949    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b\": container with ID starting with 4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b not found: ID does not exist" containerID="4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b"
Jul 22 13:18:17.116020 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.115974    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b"} err="failed to get container status \"4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b\": rpc error: code = NotFound desc = could not find container \"4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b\": container with ID starting with 4af28d6ffabf17f9155743893ea14fadd47bccceeec99b80cd32cf3cb5b2395b not found: ID does not exist"
Jul 22 13:18:17.152450 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152425    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jv79p\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-kube-api-access-jv79p\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152603 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152466    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/083bc315-a956-4431-a053-79f5d8293ad3-istio-podinfo\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152603 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152504    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/083bc315-a956-4431-a053-79f5d8293ad3-istiod-ca-cert\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152603 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152567    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-credential-socket\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152798 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152639    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-socket\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152798 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152702    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-istio-token\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152798 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152729    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-certs\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152798 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152769    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-data\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.152798 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.152793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-envoy\") pod \"083bc315-a956-4431-a053-79f5d8293ad3\" (UID: \"083bc315-a956-4431-a053-79f5d8293ad3\") "
Jul 22 13:18:17.153074 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.153018    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:18:17.153136 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.153086    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:18:17.153136 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.153104    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/083bc315-a956-4431-a053-79f5d8293ad3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:18:17.153495 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.153468    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:18:17.154100 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.154073    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-data" (OuterVolumeSpecName: "istio-data") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:18:17.154965 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.154927    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-kube-api-access-jv79p" (OuterVolumeSpecName: "kube-api-access-jv79p") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "kube-api-access-jv79p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:18:17.155066 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.155034    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-istio-token" (OuterVolumeSpecName: "istio-token") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:18:17.155611 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.155590    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:18:17.155683 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.155608    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/083bc315-a956-4431-a053-79f5d8293ad3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "083bc315-a956-4431-a053-79f5d8293ad3" (UID: "083bc315-a956-4431-a053-79f5d8293ad3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:18:17.254578 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254546    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254578 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254577    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254839 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254590    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254839 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254603    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254839 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254612    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jv79p\" (UniqueName: \"kubernetes.io/projected/083bc315-a956-4431-a053-79f5d8293ad3-kube-api-access-jv79p\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254839 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254620    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/083bc315-a956-4431-a053-79f5d8293ad3-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254839 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254629    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/083bc315-a956-4431-a053-79f5d8293ad3-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254839 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254636    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.254839 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.254644    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/083bc315-a956-4431-a053-79f5d8293ad3-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:18:17.420867 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.420831    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"]
Jul 22 13:18:17.425441 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:17.425417    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-0d04d7-openshift-default-7ffccc6899-pqq7f"]
Jul 22 13:18:18.018627 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:18.018594    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="083bc315-a956-4431-a053-79f5d8293ad3" path="/var/lib/kubelet/pods/083bc315-a956-4431-a053-79f5d8293ad3/volumes"
Jul 22 13:18:21.177597 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.177563    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"]
Jul 22 13:18:21.178014 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.177999    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy"
Jul 22 13:18:21.178059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.178016    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy"
Jul 22 13:18:21.178059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.178025    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy"
Jul 22 13:18:21.178059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.178031    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy"
Jul 22 13:18:21.178151 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.178097    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy"
Jul 22 13:18:21.178151 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.178107    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="083bc315-a956-4431-a053-79f5d8293ad3" containerName="istio-proxy"
Jul 22 13:18:21.182640 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.182620    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.189910 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.189893    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-494b12-openshift-default-dockercfg-wlpbk\""
Jul 22 13:18:21.203393 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.203354    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"]
Jul 22 13:18:21.293379 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-token\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293595 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293396    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-certs\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293595 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293426    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-envoy\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293595 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9067401-0f27-4bdc-91df-8d773093fdb2-istiod-ca-cert\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293595 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293576    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-socket\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-podinfo\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293645    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-data\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293707    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-credential-socket\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.293866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.293766    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgqhz\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-kube-api-access-pgqhz\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395209 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-credential-socket\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395398 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395214    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgqhz\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-kube-api-access-pgqhz\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395398 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-token\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395398 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395316    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-certs\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395398 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395340    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-envoy\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395398 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395371    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9067401-0f27-4bdc-91df-8d773093fdb2-istiod-ca-cert\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395669 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395412    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-socket\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395669 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395450    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-podinfo\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395669 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-data\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395861 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-certs\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395861 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395770    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-credential-socket\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395928 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-socket\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.395989 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.395964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-data\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.396147 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.396074    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9067401-0f27-4bdc-91df-8d773093fdb2-istiod-ca-cert\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.397820 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.397797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-envoy\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.398069 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.398052    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-podinfo\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.404187 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.404166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-token\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.404414 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.404393    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgqhz\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-kube-api-access-pgqhz\") pod \"e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") " pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.492998 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.492957    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:21.632913 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.632885    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"]
Jul 22 13:18:21.635324 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:18:21.635295    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9067401_0f27_4bdc_91df_8d773093fdb2.slice/crio-f5e6acd9c92623b52ec3480c8666f484c0a03a2d1aeeee2f1852d4639bbbdea7 WatchSource:0}: Error finding container f5e6acd9c92623b52ec3480c8666f484c0a03a2d1aeeee2f1852d4639bbbdea7: Status 404 returned error can't find the container with id f5e6acd9c92623b52ec3480c8666f484c0a03a2d1aeeee2f1852d4639bbbdea7
Jul 22 13:18:21.637960 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.637921    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:21.638029 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.638009    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:21.638098 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:21.638080    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:22.125073 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:22.125037    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" event={"ID":"c9067401-0f27-4bdc-91df-8d773093fdb2","Type":"ContainerStarted","Data":"1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13"}
Jul 22 13:18:22.125251 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:22.125091    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" event={"ID":"c9067401-0f27-4bdc-91df-8d773093fdb2","Type":"ContainerStarted","Data":"f5e6acd9c92623b52ec3480c8666f484c0a03a2d1aeeee2f1852d4639bbbdea7"}
Jul 22 13:18:22.148706 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:22.148654    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" podStartSLOduration=1.148638923 podStartE2EDuration="1.148638923s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:18:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:18:22.146819034 +0000 UTC m=+2486.611717083" watchObservedRunningTime="2026-07-22 13:18:22.148638923 +0000 UTC m=+2486.613536973"
Jul 22 13:18:22.493284 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:22.493249    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:22.498150 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:22.498120    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:23.130579 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:23.130545    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:23.131459 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:23.131439    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:31.129926 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:31.129880    2563 patch_prober.go:28] interesting pod/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 22 13:18:31.130379 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:31.129965    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 22 13:18:32.181732 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:32.181693    2563 generic.go:356] "Generic (PLEG): container finished" podID="05794723-c5b4-41e2-bc62-191dfa59771f" containerID="f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5" exitCode=0
Jul 22 13:18:32.182174 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:32.181785    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" event={"ID":"05794723-c5b4-41e2-bc62-191dfa59771f","Type":"ContainerDied","Data":"f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5"}
Jul 22 13:18:32.182270 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:32.182253    2563 scope.go:117] "RemoveContainer" containerID="f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5"
Jul 22 13:18:32.183323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:32.183290    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:32.183624 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:32.183361    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:32.183624 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:32.183392    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:33.193341 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:33.192347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" event={"ID":"05794723-c5b4-41e2-bc62-191dfa59771f","Type":"ContainerStarted","Data":"8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc"}
Jul 22 13:18:33.979256 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:33.979210    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:33.979518 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:33.979486    2563 patch_prober.go:28] interesting pod/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 22 13:18:33.979793 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:33.979731    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 22 13:18:34.979504 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:34.979469    2563 patch_prober.go:28] interesting pod/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 22 13:18:34.980002 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:34.979549    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 22 13:18:35.980167 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:35.980130    2563 patch_prober.go:28] interesting pod/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 22 13:18:35.980580 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:35.980198    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 22 13:18:36.979712 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:36.979665    2563 patch_prober.go:28] interesting pod/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body=
Jul 22 13:18:36.979947 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:36.979737    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused"
Jul 22 13:18:37.983086 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:37.983059    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:37.983495 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:37.983356    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:37.984033 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:37.984016    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:18:38.131954 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.131870    2563 patch_prober.go:28] interesting pod/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 22 13:18:38.131954 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.131924    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 22 13:18:38.222809 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.222701    2563 generic.go:356] "Generic (PLEG): container finished" podID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerID="1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13" exitCode=0
Jul 22 13:18:38.222809 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.222784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" event={"ID":"c9067401-0f27-4bdc-91df-8d773093fdb2","Type":"ContainerDied","Data":"1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13"}
Jul 22 13:18:38.223260 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.223245    2563 scope.go:117] "RemoveContainer" containerID="1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13"
Jul 22 13:18:38.224189 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.224166    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:38.224252 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.224228    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:38.224293 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:38.224261    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"}
Jul 22 13:18:39.230213 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:39.230163    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" event={"ID":"c9067401-0f27-4bdc-91df-8d773093fdb2","Type":"ContainerStarted","Data":"3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87"}
Jul 22 13:18:39.493624 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:39.493532    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:39.493835 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:39.493729    2563 patch_prober.go:28] interesting pod/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 22 13:18:39.493835 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:39.493809    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 22 13:18:40.494087 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:40.494050    2563 patch_prober.go:28] interesting pod/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 22 13:18:40.494557 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:40.494114    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 22 13:18:41.494336 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:41.494297    2563 patch_prober.go:28] interesting pod/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 22 13:18:41.494733 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:41.494358    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 22 13:18:42.493421 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.493381    2563 patch_prober.go:28] interesting pod/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body=
Jul 22 13:18:42.493596 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.493448    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused"
Jul 22 13:18:42.535434 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.535404    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"]
Jul 22 13:18:42.538041 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.538022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.540845 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.540823    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"openshift-service-ca.crt\""
Jul 22 13:18:42.541926 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.541899    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"kube-root-ca.crt\""
Jul 22 13:18:42.542058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.541998    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"auth-test-model-634440-kserve-self-signed-certs\""
Jul 22 13:18:42.542058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.541902    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"default-dockercfg-t45sl\""
Jul 22 13:18:42.548913 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.548891    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"]
Jul 22 13:18:42.608875 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.608837    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-home\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.609058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.608887    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-model-cache\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.609058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.608969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-tmp-dir\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.609058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.608999    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk7nf\" (UniqueName: \"kubernetes.io/projected/ec1108ce-b281-443d-b86c-192e379a836c-kube-api-access-sk7nf\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.609058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.609033    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-dshm\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.609058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.609047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec1108ce-b281-443d-b86c-192e379a836c-tls-certs\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.709605 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.709568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-home\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.709821 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.709617    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-model-cache\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.709821 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.709651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-tmp-dir\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.709821 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.709667    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sk7nf\" (UniqueName: \"kubernetes.io/projected/ec1108ce-b281-443d-b86c-192e379a836c-kube-api-access-sk7nf\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.709821 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.709697    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-dshm\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.709821 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.709716    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec1108ce-b281-443d-b86c-192e379a836c-tls-certs\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.710104 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.710068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-home\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.710153 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.710110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-tmp-dir\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.710190 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.710146    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-model-cache\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.712199 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.712173    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-dshm\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.712318 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.712301    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec1108ce-b281-443d-b86c-192e379a836c-tls-certs\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.718041 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.718022    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk7nf\" (UniqueName: \"kubernetes.io/projected/ec1108ce-b281-443d-b86c-192e379a836c-kube-api-access-sk7nf\") pod \"auth-test-model-634440-kserve-dbc488775-5swfz\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") " pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:42.849202 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:42.849113    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:43.033671 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:43.033643    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"]
Jul 22 13:18:43.035403 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:18:43.035375    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec1108ce_b281_443d_b86c_192e379a836c.slice/crio-a60a51718aee2eae72ad898dc91fb2cbb86c6b0e77cbb23de6c19e47f6a84660 WatchSource:0}: Error finding container a60a51718aee2eae72ad898dc91fb2cbb86c6b0e77cbb23de6c19e47f6a84660: Status 404 returned error can't find the container with id a60a51718aee2eae72ad898dc91fb2cbb86c6b0e77cbb23de6c19e47f6a84660
Jul 22 13:18:43.251314 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:43.251271    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz" event={"ID":"ec1108ce-b281-443d-b86c-192e379a836c","Type":"ContainerStarted","Data":"b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d"}
Jul 22 13:18:43.251520 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:43.251328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz" event={"ID":"ec1108ce-b281-443d-b86c-192e379a836c","Type":"ContainerStarted","Data":"a60a51718aee2eae72ad898dc91fb2cbb86c6b0e77cbb23de6c19e47f6a84660"}
Jul 22 13:18:43.273110 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:43.273044    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz" podStartSLOduration=1.273023023 podStartE2EDuration="1.273023023s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:18:43.269886971 +0000 UTC m=+2507.734785021" watchObservedRunningTime="2026-07-22 13:18:43.273023023 +0000 UTC m=+2507.737921073"
Jul 22 13:18:43.497868 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:43.497835    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:43.498220 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:43.498198    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:43.498992 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:43.498973    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:18:52.849694 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:52.849613    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:52.849694 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:52.849664    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:52.867259 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:52.867224    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:18:53.310899 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:18:53.310872    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:19:16.718267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.718227    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"]
Jul 22 13:19:16.721051 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.721032    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.723557 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.723536    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"kube-root-ca.crt\""
Jul 22 13:19:16.724706 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.724689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"auth-test-model-494b12-kserve-self-signed-certs\""
Jul 22 13:19:16.724833 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.724689    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"default-dockercfg-l9khj\""
Jul 22 13:19:16.724833 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.724739    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"openshift-service-ca.crt\""
Jul 22 13:19:16.735555 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.735527    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"]
Jul 22 13:19:16.822800 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.822762    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-home\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.823000 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.822824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhsx4\" (UniqueName: \"kubernetes.io/projected/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-kube-api-access-lhsx4\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.823000 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.822863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tmp-dir\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.823000 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.822974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tls-certs\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.823159 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.823001    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-model-cache\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.823159 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.823047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-dshm\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.924693 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.924652    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tls-certs\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.924925 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.924718    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-model-cache\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.924925 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.924799    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-dshm\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.924925 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.924843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-home\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.924925 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.924905    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhsx4\" (UniqueName: \"kubernetes.io/projected/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-kube-api-access-lhsx4\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.925158 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.924952    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tmp-dir\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.925210 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.925168    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-model-cache\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.925379 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.925350    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tmp-dir\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.926062 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.926039    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-home\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.928053 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.928015    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tls-certs\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.928645 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.928628    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-dshm\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:16.934367 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:16.934342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhsx4\" (UniqueName: \"kubernetes.io/projected/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-kube-api-access-lhsx4\") pod \"auth-test-model-494b12-kserve-8c6f45fc9-8lmv4\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") " pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:17.038816 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:17.038685    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:17.173771 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:17.173664    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"]
Jul 22 13:19:17.176580 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:19:17.176545    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e3d39e5_c139_46ee_b3cb_41b140c03ad8.slice/crio-1ea1a13edb172e0b208b87331afcd79f6d5c45f4c858b59cfe8095678a31ba4c WatchSource:0}: Error finding container 1ea1a13edb172e0b208b87331afcd79f6d5c45f4c858b59cfe8095678a31ba4c: Status 404 returned error can't find the container with id 1ea1a13edb172e0b208b87331afcd79f6d5c45f4c858b59cfe8095678a31ba4c
Jul 22 13:19:17.415181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:17.415098    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4" event={"ID":"4e3d39e5-c139-46ee-b3cb-41b140c03ad8","Type":"ContainerStarted","Data":"25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa"}
Jul 22 13:19:17.415181 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:17.415136    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4" event={"ID":"4e3d39e5-c139-46ee-b3cb-41b140c03ad8","Type":"ContainerStarted","Data":"1ea1a13edb172e0b208b87331afcd79f6d5c45f4c858b59cfe8095678a31ba4c"}
Jul 22 13:19:17.433717 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:17.433660    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4" podStartSLOduration=1.4336432239999999 podStartE2EDuration="1.433643224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:19:17.432358756 +0000 UTC m=+2541.897256829" watchObservedRunningTime="2026-07-22 13:19:17.433643224 +0000 UTC m=+2541.898541272"
Jul 22 13:19:27.039263 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:27.039218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:27.039796 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:27.039281    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:27.052203 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:27.052181    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:19:27.472272 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:19:27.472244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:21:18.759915 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:18.759879    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"]
Jul 22 13:21:19.703667 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:19.703631    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"]
Jul 22 13:21:33.780631 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:33.780555    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz" podUID="ec1108ce-b281-443d-b86c-192e379a836c" containerName="main" containerID="cri-o://b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d" gracePeriod=45
Jul 22 13:21:34.036454 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.036392    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:21:34.079552 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.079517    2563 generic.go:356] "Generic (PLEG): container finished" podID="ec1108ce-b281-443d-b86c-192e379a836c" containerID="b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d" exitCode=0
Jul 22 13:21:34.079793 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.079564    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"
Jul 22 13:21:34.079793 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.079616    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz" event={"ID":"ec1108ce-b281-443d-b86c-192e379a836c","Type":"ContainerDied","Data":"b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d"}
Jul 22 13:21:34.079793 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.079672    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz" event={"ID":"ec1108ce-b281-443d-b86c-192e379a836c","Type":"ContainerDied","Data":"a60a51718aee2eae72ad898dc91fb2cbb86c6b0e77cbb23de6c19e47f6a84660"}
Jul 22 13:21:34.079793 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.079721    2563 scope.go:117] "RemoveContainer" containerID="b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d"
Jul 22 13:21:34.088651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.088634    2563 scope.go:117] "RemoveContainer" containerID="b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d"
Jul 22 13:21:34.088934 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:21:34.088914    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d\": container with ID starting with b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d not found: ID does not exist" containerID="b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d"
Jul 22 13:21:34.088991 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.088942    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d"} err="failed to get container status \"b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d\": rpc error: code = NotFound desc = could not find container \"b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d\": container with ID starting with b6cad322209c0d3748e6a6eb017ec979f264c230ece0ec8417cb09cbd8e1356d not found: ID does not exist"
Jul 22 13:21:34.117390 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117368    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec1108ce-b281-443d-b86c-192e379a836c-tls-certs\") pod \"ec1108ce-b281-443d-b86c-192e379a836c\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") "
Jul 22 13:21:34.117483 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117406    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-home\") pod \"ec1108ce-b281-443d-b86c-192e379a836c\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") "
Jul 22 13:21:34.117483 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117455    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-tmp-dir\") pod \"ec1108ce-b281-443d-b86c-192e379a836c\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") "
Jul 22 13:21:34.117564 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117500    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-model-cache\") pod \"ec1108ce-b281-443d-b86c-192e379a836c\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") "
Jul 22 13:21:34.117564 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117546    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sk7nf\" (UniqueName: \"kubernetes.io/projected/ec1108ce-b281-443d-b86c-192e379a836c-kube-api-access-sk7nf\") pod \"ec1108ce-b281-443d-b86c-192e379a836c\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") "
Jul 22 13:21:34.117653 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117601    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-dshm\") pod \"ec1108ce-b281-443d-b86c-192e379a836c\" (UID: \"ec1108ce-b281-443d-b86c-192e379a836c\") "
Jul 22 13:21:34.117730 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117708    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-home" (OuterVolumeSpecName: "home") pod "ec1108ce-b281-443d-b86c-192e379a836c" (UID: "ec1108ce-b281-443d-b86c-192e379a836c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:34.117818 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117726    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ec1108ce-b281-443d-b86c-192e379a836c" (UID: "ec1108ce-b281-443d-b86c-192e379a836c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:34.117878 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117819    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-model-cache" (OuterVolumeSpecName: "model-cache") pod "ec1108ce-b281-443d-b86c-192e379a836c" (UID: "ec1108ce-b281-443d-b86c-192e379a836c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:34.117976 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117958    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:34.118045 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117977    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:34.118045 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.117985    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:34.119708 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.119680    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec1108ce-b281-443d-b86c-192e379a836c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ec1108ce-b281-443d-b86c-192e379a836c" (UID: "ec1108ce-b281-443d-b86c-192e379a836c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:21:34.119833 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.119806    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec1108ce-b281-443d-b86c-192e379a836c-kube-api-access-sk7nf" (OuterVolumeSpecName: "kube-api-access-sk7nf") pod "ec1108ce-b281-443d-b86c-192e379a836c" (UID: "ec1108ce-b281-443d-b86c-192e379a836c"). InnerVolumeSpecName "kube-api-access-sk7nf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:21:34.119898 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.119877    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-dshm" (OuterVolumeSpecName: "dshm") pod "ec1108ce-b281-443d-b86c-192e379a836c" (UID: "ec1108ce-b281-443d-b86c-192e379a836c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:34.218699 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.218663    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sk7nf\" (UniqueName: \"kubernetes.io/projected/ec1108ce-b281-443d-b86c-192e379a836c-kube-api-access-sk7nf\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:34.218699 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.218695    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ec1108ce-b281-443d-b86c-192e379a836c-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:34.218699 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.218707    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ec1108ce-b281-443d-b86c-192e379a836c-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:34.401643 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.401614    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"]
Jul 22 13:21:34.405155 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.405129    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-634440/auth-test-model-634440-kserve-dbc488775-5swfz"]
Jul 22 13:21:34.724644 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.724586    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4" podUID="4e3d39e5-c139-46ee-b3cb-41b140c03ad8" containerName="main" containerID="cri-o://25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa" gracePeriod=45
Jul 22 13:21:34.979294 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:34.979235    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:21:35.026898 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.026872    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhsx4\" (UniqueName: \"kubernetes.io/projected/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-kube-api-access-lhsx4\") pod \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") "
Jul 22 13:21:35.027059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.026914    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-dshm\") pod \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") "
Jul 22 13:21:35.027059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.026991    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-home\") pod \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") "
Jul 22 13:21:35.027059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027021    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-model-cache\") pod \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") "
Jul 22 13:21:35.027059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027042    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tls-certs\") pod \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") "
Jul 22 13:21:35.027271 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027093    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tmp-dir\") pod \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\" (UID: \"4e3d39e5-c139-46ee-b3cb-41b140c03ad8\") "
Jul 22 13:21:35.027331 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027280    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-model-cache" (OuterVolumeSpecName: "model-cache") pod "4e3d39e5-c139-46ee-b3cb-41b140c03ad8" (UID: "4e3d39e5-c139-46ee-b3cb-41b140c03ad8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:35.027331 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027304    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-home" (OuterVolumeSpecName: "home") pod "4e3d39e5-c139-46ee-b3cb-41b140c03ad8" (UID: "4e3d39e5-c139-46ee-b3cb-41b140c03ad8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:35.027431 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027396    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4e3d39e5-c139-46ee-b3cb-41b140c03ad8" (UID: "4e3d39e5-c139-46ee-b3cb-41b140c03ad8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:35.027474 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027433    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:35.027474 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.027452    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:35.029277 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.029259    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-dshm" (OuterVolumeSpecName: "dshm") pod "4e3d39e5-c139-46ee-b3cb-41b140c03ad8" (UID: "4e3d39e5-c139-46ee-b3cb-41b140c03ad8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:21:35.029717 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.029696    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4e3d39e5-c139-46ee-b3cb-41b140c03ad8" (UID: "4e3d39e5-c139-46ee-b3cb-41b140c03ad8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:21:35.029842 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.029715    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-kube-api-access-lhsx4" (OuterVolumeSpecName: "kube-api-access-lhsx4") pod "4e3d39e5-c139-46ee-b3cb-41b140c03ad8" (UID: "4e3d39e5-c139-46ee-b3cb-41b140c03ad8"). InnerVolumeSpecName "kube-api-access-lhsx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:21:35.085761 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.085723    2563 generic.go:356] "Generic (PLEG): container finished" podID="4e3d39e5-c139-46ee-b3cb-41b140c03ad8" containerID="25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa" exitCode=0
Jul 22 13:21:35.085919 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.085789    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"
Jul 22 13:21:35.085919 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.085808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4" event={"ID":"4e3d39e5-c139-46ee-b3cb-41b140c03ad8","Type":"ContainerDied","Data":"25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa"}
Jul 22 13:21:35.085919 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.085847    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4" event={"ID":"4e3d39e5-c139-46ee-b3cb-41b140c03ad8","Type":"ContainerDied","Data":"1ea1a13edb172e0b208b87331afcd79f6d5c45f4c858b59cfe8095678a31ba4c"}
Jul 22 13:21:35.085919 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.085865    2563 scope.go:117] "RemoveContainer" containerID="25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa"
Jul 22 13:21:35.094567 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.094550    2563 scope.go:117] "RemoveContainer" containerID="25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa"
Jul 22 13:21:35.094837 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:21:35.094818    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa\": container with ID starting with 25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa not found: ID does not exist" containerID="25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa"
Jul 22 13:21:35.094889 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.094845    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa"} err="failed to get container status \"25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa\": rpc error: code = NotFound desc = could not find container \"25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa\": container with ID starting with 25697edb7c326d24209643260b41e30f288108ca008290c15c8cced2672555fa not found: ID does not exist"
Jul 22 13:21:35.108505 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.108485    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"]
Jul 22 13:21:35.113876 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.113857    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/auth-test-model-494b12-kserve-8c6f45fc9-8lmv4"]
Jul 22 13:21:35.128241 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.128222    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhsx4\" (UniqueName: \"kubernetes.io/projected/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-kube-api-access-lhsx4\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:35.128323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.128245    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:35.128323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.128260    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:35.128323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:35.128273    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4e3d39e5-c139-46ee-b3cb-41b140c03ad8-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:21:36.019187 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:36.019157    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e3d39e5-c139-46ee-b3cb-41b140c03ad8" path="/var/lib/kubelet/pods/4e3d39e5-c139-46ee-b3cb-41b140c03ad8/volumes"
Jul 22 13:21:36.019589 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:36.019573    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec1108ce-b281-443d-b86c-192e379a836c" path="/var/lib/kubelet/pods/ec1108ce-b281-443d-b86c-192e379a836c/volumes"
Jul 22 13:21:40.323304 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.323273    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"]
Jul 22 13:21:40.324052 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.323634    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec1108ce-b281-443d-b86c-192e379a836c" containerName="main"
Jul 22 13:21:40.324052 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.323645    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec1108ce-b281-443d-b86c-192e379a836c" containerName="main"
Jul 22 13:21:40.324052 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.323666    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e3d39e5-c139-46ee-b3cb-41b140c03ad8" containerName="main"
Jul 22 13:21:40.324052 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.323671    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e3d39e5-c139-46ee-b3cb-41b140c03ad8" containerName="main"
Jul 22 13:21:40.324052 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.323732    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e3d39e5-c139-46ee-b3cb-41b140c03ad8" containerName="main"
Jul 22 13:21:40.324052 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.323739    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec1108ce-b281-443d-b86c-192e379a836c" containerName="main"
Jul 22 13:21:40.326621 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.326604    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.329390 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.329370    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"openshift-service-ca.crt\""
Jul 22 13:21:40.330765 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.330730    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"sub-test-model-634440-kserve-self-signed-certs\""
Jul 22 13:21:40.330866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.330766    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"kube-root-ca.crt\""
Jul 22 13:21:40.330866 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.330818    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"default-dockercfg-t45sl\""
Jul 22 13:21:40.337267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.337246    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"]
Jul 22 13:21:40.375521 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.375491    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-model-cache\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.375651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.375538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-home\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.375651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.375558    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxhr8\" (UniqueName: \"kubernetes.io/projected/a0a95514-9ac3-49b9-8f4b-92bc237ae995-kube-api-access-cxhr8\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.375651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.375581    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tmp-dir\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.375651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.375621    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-dshm\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.375651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.375636    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tls-certs\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476158 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-model-cache\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476347 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476183    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-home\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476347 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476208    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxhr8\" (UniqueName: \"kubernetes.io/projected/a0a95514-9ac3-49b9-8f4b-92bc237ae995-kube-api-access-cxhr8\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476347 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476227    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tmp-dir\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476347 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476261    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-dshm\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476596 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476403    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tls-certs\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476596 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476572    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-model-cache\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476683 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476619    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-home\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.476855 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.476834    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tmp-dir\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.478540 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.478521    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-dshm\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.478855 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.478836    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tls-certs\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.485518 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.485496    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxhr8\" (UniqueName: \"kubernetes.io/projected/a0a95514-9ac3-49b9-8f4b-92bc237ae995-kube-api-access-cxhr8\") pod \"sub-test-model-634440-kserve-64d87bfb97-j4vbw\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") " pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.637075 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.636974    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:40.771394 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:40.771256    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"]
Jul 22 13:21:40.773835 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:21:40.773806    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda0a95514_9ac3_49b9_8f4b_92bc237ae995.slice/crio-06fa64faf0926085cb0cf4f6ddc989ae24aabc8bfd79c15b37e8b60c8a422409 WatchSource:0}: Error finding container 06fa64faf0926085cb0cf4f6ddc989ae24aabc8bfd79c15b37e8b60c8a422409: Status 404 returned error can't find the container with id 06fa64faf0926085cb0cf4f6ddc989ae24aabc8bfd79c15b37e8b60c8a422409
Jul 22 13:21:41.118315 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:41.118278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw" event={"ID":"a0a95514-9ac3-49b9-8f4b-92bc237ae995","Type":"ContainerStarted","Data":"00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571"}
Jul 22 13:21:41.118518 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:41.118328    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw" event={"ID":"a0a95514-9ac3-49b9-8f4b-92bc237ae995","Type":"ContainerStarted","Data":"06fa64faf0926085cb0cf4f6ddc989ae24aabc8bfd79c15b37e8b60c8a422409"}
Jul 22 13:21:41.137209 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:41.137155    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw" podStartSLOduration=1.137140022 podStartE2EDuration="1.137140022s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:21:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:21:41.135888245 +0000 UTC m=+2685.600786293" watchObservedRunningTime="2026-07-22 13:21:41.137140022 +0000 UTC m=+2685.602038098"
Jul 22 13:21:50.637638 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:50.637551    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:50.637638 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:50.637618    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:50.649567 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:50.649545    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:21:51.185323 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:21:51.185290    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:22:02.802056 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.802016    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"]
Jul 22 13:22:02.804560 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.804531    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.807300 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.807277    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"openshift-service-ca.crt\""
Jul 22 13:22:02.808711 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.808697    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"kube-root-ca.crt\""
Jul 22 13:22:02.808794 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.808721    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"default-dockercfg-l9khj\""
Jul 22 13:22:02.808794 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.808740    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"sub-test-model-494b12-kserve-self-signed-certs\""
Jul 22 13:22:02.815457 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.815433    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"]
Jul 22 13:22:02.890570 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.890538    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-dshm\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.890570 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.890574    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-model-cache\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.890846 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.890605    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-tmp-dir\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.890846 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.890676    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/778937c2-1c55-4007-a43e-e859ee103e27-tls-certs\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.890846 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.890802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-home\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.891001 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.890860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6rmr\" (UniqueName: \"kubernetes.io/projected/778937c2-1c55-4007-a43e-e859ee103e27-kube-api-access-b6rmr\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.991815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.991783    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-dshm\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.991815 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.991819    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-model-cache\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.992064 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.991846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-tmp-dir\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.992064 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.991963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/778937c2-1c55-4007-a43e-e859ee103e27-tls-certs\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.992064 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.992043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-home\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.992228 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.992092    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6rmr\" (UniqueName: \"kubernetes.io/projected/778937c2-1c55-4007-a43e-e859ee103e27-kube-api-access-b6rmr\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.992283 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.992238    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-model-cache\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.992283 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.992250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-tmp-dir\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.992365 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.992330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-home\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.994372 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.994353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-dshm\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:02.994534 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:02.994514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/778937c2-1c55-4007-a43e-e859ee103e27-tls-certs\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:03.001527 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:03.001506    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6rmr\" (UniqueName: \"kubernetes.io/projected/778937c2-1c55-4007-a43e-e859ee103e27-kube-api-access-b6rmr\") pod \"sub-test-model-494b12-kserve-686bcf8d75-tnjfh\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") " pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:03.114684 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:03.114611    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:03.249390 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:03.249341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"]
Jul 22 13:22:03.252328 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:22:03.252295    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod778937c2_1c55_4007_a43e_e859ee103e27.slice/crio-8e1b631dc0ab12c33d9ccc754aa42a4c9bab6a5eda8dd1f2d93439899432b611 WatchSource:0}: Error finding container 8e1b631dc0ab12c33d9ccc754aa42a4c9bab6a5eda8dd1f2d93439899432b611: Status 404 returned error can't find the container with id 8e1b631dc0ab12c33d9ccc754aa42a4c9bab6a5eda8dd1f2d93439899432b611
Jul 22 13:22:04.238224 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:04.238189    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh" event={"ID":"778937c2-1c55-4007-a43e-e859ee103e27","Type":"ContainerStarted","Data":"1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68"}
Jul 22 13:22:04.238224 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:04.238232    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh" event={"ID":"778937c2-1c55-4007-a43e-e859ee103e27","Type":"ContainerStarted","Data":"8e1b631dc0ab12c33d9ccc754aa42a4c9bab6a5eda8dd1f2d93439899432b611"}
Jul 22 13:22:04.257886 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:04.257842    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh" podStartSLOduration=2.257826813 podStartE2EDuration="2.257826813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:22:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:22:04.257079493 +0000 UTC m=+2708.721977552" watchObservedRunningTime="2026-07-22 13:22:04.257826813 +0000 UTC m=+2708.722724861"
Jul 22 13:22:13.115771 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:13.115704    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:13.115771 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:13.115787    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:13.128631 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:13.128600    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:13.292609 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:13.292580    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:22:44.719355 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:44.719322    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"]
Jul 22 13:22:45.598247 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:45.598217    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"]
Jul 22 13:22:59.741931 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:22:59.741876    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw" podUID="a0a95514-9ac3-49b9-8f4b-92bc237ae995" containerName="main" containerID="cri-o://00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571" gracePeriod=45
Jul 22 13:23:00.002397 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.002327    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:23:00.129757 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.129713    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tls-certs\") pod \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") "
Jul 22 13:23:00.129936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.129772    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tmp-dir\") pod \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") "
Jul 22 13:23:00.129936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.129800    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-home\") pod \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") "
Jul 22 13:23:00.129936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.129849    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-dshm\") pod \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") "
Jul 22 13:23:00.129936 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.129882    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxhr8\" (UniqueName: \"kubernetes.io/projected/a0a95514-9ac3-49b9-8f4b-92bc237ae995-kube-api-access-cxhr8\") pod \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") "
Jul 22 13:23:00.130122 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.129937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-model-cache\") pod \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\" (UID: \"a0a95514-9ac3-49b9-8f4b-92bc237ae995\") "
Jul 22 13:23:00.130122 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.130091    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-home" (OuterVolumeSpecName: "home") pod "a0a95514-9ac3-49b9-8f4b-92bc237ae995" (UID: "a0a95514-9ac3-49b9-8f4b-92bc237ae995"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:00.130122 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.130105    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a0a95514-9ac3-49b9-8f4b-92bc237ae995" (UID: "a0a95514-9ac3-49b9-8f4b-92bc237ae995"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:00.130287 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.130264    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:00.130287 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.130285    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:00.130378 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.130283    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-model-cache" (OuterVolumeSpecName: "model-cache") pod "a0a95514-9ac3-49b9-8f4b-92bc237ae995" (UID: "a0a95514-9ac3-49b9-8f4b-92bc237ae995"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:00.131975 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.131951    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a0a95514-9ac3-49b9-8f4b-92bc237ae995" (UID: "a0a95514-9ac3-49b9-8f4b-92bc237ae995"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:23:00.132482 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.132463    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-dshm" (OuterVolumeSpecName: "dshm") pod "a0a95514-9ac3-49b9-8f4b-92bc237ae995" (UID: "a0a95514-9ac3-49b9-8f4b-92bc237ae995"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:00.132584 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.132489    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0a95514-9ac3-49b9-8f4b-92bc237ae995-kube-api-access-cxhr8" (OuterVolumeSpecName: "kube-api-access-cxhr8") pod "a0a95514-9ac3-49b9-8f4b-92bc237ae995" (UID: "a0a95514-9ac3-49b9-8f4b-92bc237ae995"). InnerVolumeSpecName "kube-api-access-cxhr8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:23:00.231204 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.231167    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:00.231204 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.231198    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxhr8\" (UniqueName: \"kubernetes.io/projected/a0a95514-9ac3-49b9-8f4b-92bc237ae995-kube-api-access-cxhr8\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:00.231204 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.231210    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0a95514-9ac3-49b9-8f4b-92bc237ae995-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:00.231204 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.231218    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0a95514-9ac3-49b9-8f4b-92bc237ae995-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:00.511243 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.511207    2563 generic.go:356] "Generic (PLEG): container finished" podID="a0a95514-9ac3-49b9-8f4b-92bc237ae995" containerID="00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571" exitCode=0
Jul 22 13:23:00.511444 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.511270    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"
Jul 22 13:23:00.511444 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.511286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw" event={"ID":"a0a95514-9ac3-49b9-8f4b-92bc237ae995","Type":"ContainerDied","Data":"00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571"}
Jul 22 13:23:00.511444 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.511316    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw" event={"ID":"a0a95514-9ac3-49b9-8f4b-92bc237ae995","Type":"ContainerDied","Data":"06fa64faf0926085cb0cf4f6ddc989ae24aabc8bfd79c15b37e8b60c8a422409"}
Jul 22 13:23:00.511444 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.511335    2563 scope.go:117] "RemoveContainer" containerID="00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571"
Jul 22 13:23:00.521191 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.521173    2563 scope.go:117] "RemoveContainer" containerID="00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571"
Jul 22 13:23:00.521445 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:23:00.521428    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571\": container with ID starting with 00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571 not found: ID does not exist" containerID="00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571"
Jul 22 13:23:00.521485 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.521454    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571"} err="failed to get container status \"00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571\": rpc error: code = NotFound desc = could not find container \"00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571\": container with ID starting with 00698ff4bc65023fca1b096b59eb2312314e1be0734ae325ec82001ed616f571 not found: ID does not exist"
Jul 22 13:23:00.532019 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.531985    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"]
Jul 22 13:23:00.538016 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.537995    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-634440/sub-test-model-634440-kserve-64d87bfb97-j4vbw"]
Jul 22 13:23:00.619364 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.619315    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh" podUID="778937c2-1c55-4007-a43e-e859ee103e27" containerName="main" containerID="cri-o://1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68" gracePeriod=45
Jul 22 13:23:00.884929 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.884900    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:23:00.938413 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938386    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/778937c2-1c55-4007-a43e-e859ee103e27-tls-certs\") pod \"778937c2-1c55-4007-a43e-e859ee103e27\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") "
Jul 22 13:23:00.938581 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938432    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-tmp-dir\") pod \"778937c2-1c55-4007-a43e-e859ee103e27\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") "
Jul 22 13:23:00.938581 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6rmr\" (UniqueName: \"kubernetes.io/projected/778937c2-1c55-4007-a43e-e859ee103e27-kube-api-access-b6rmr\") pod \"778937c2-1c55-4007-a43e-e859ee103e27\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") "
Jul 22 13:23:00.938581 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938548    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-home\") pod \"778937c2-1c55-4007-a43e-e859ee103e27\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") "
Jul 22 13:23:00.938735 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938588    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-dshm\") pod \"778937c2-1c55-4007-a43e-e859ee103e27\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") "
Jul 22 13:23:00.938735 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938604    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-model-cache\") pod \"778937c2-1c55-4007-a43e-e859ee103e27\" (UID: \"778937c2-1c55-4007-a43e-e859ee103e27\") "
Jul 22 13:23:00.938735 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938710    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "778937c2-1c55-4007-a43e-e859ee103e27" (UID: "778937c2-1c55-4007-a43e-e859ee103e27"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:00.938918 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938900    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-home" (OuterVolumeSpecName: "home") pod "778937c2-1c55-4007-a43e-e859ee103e27" (UID: "778937c2-1c55-4007-a43e-e859ee103e27"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:00.938979 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938924    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-model-cache" (OuterVolumeSpecName: "model-cache") pod "778937c2-1c55-4007-a43e-e859ee103e27" (UID: "778937c2-1c55-4007-a43e-e859ee103e27"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:00.938979 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.938971    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:00.940826 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.940791    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/778937c2-1c55-4007-a43e-e859ee103e27-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "778937c2-1c55-4007-a43e-e859ee103e27" (UID: "778937c2-1c55-4007-a43e-e859ee103e27"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:23:00.940928 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.940866    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/778937c2-1c55-4007-a43e-e859ee103e27-kube-api-access-b6rmr" (OuterVolumeSpecName: "kube-api-access-b6rmr") pod "778937c2-1c55-4007-a43e-e859ee103e27" (UID: "778937c2-1c55-4007-a43e-e859ee103e27"). InnerVolumeSpecName "kube-api-access-b6rmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:23:00.940928 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:00.940880    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-dshm" (OuterVolumeSpecName: "dshm") pod "778937c2-1c55-4007-a43e-e859ee103e27" (UID: "778937c2-1c55-4007-a43e-e859ee103e27"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:23:01.040154 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.040116    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:01.040154 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.040146    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:01.040368 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.040158    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/778937c2-1c55-4007-a43e-e859ee103e27-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:01.040368 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.040181    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/778937c2-1c55-4007-a43e-e859ee103e27-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:01.040368 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.040194    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6rmr\" (UniqueName: \"kubernetes.io/projected/778937c2-1c55-4007-a43e-e859ee103e27-kube-api-access-b6rmr\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:23:01.517964 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.517928    2563 generic.go:356] "Generic (PLEG): container finished" podID="778937c2-1c55-4007-a43e-e859ee103e27" containerID="1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68" exitCode=0
Jul 22 13:23:01.518146 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.517970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"
Jul 22 13:23:01.518146 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.517970    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh" event={"ID":"778937c2-1c55-4007-a43e-e859ee103e27","Type":"ContainerDied","Data":"1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68"}
Jul 22 13:23:01.518146 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.518094    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh" event={"ID":"778937c2-1c55-4007-a43e-e859ee103e27","Type":"ContainerDied","Data":"8e1b631dc0ab12c33d9ccc754aa42a4c9bab6a5eda8dd1f2d93439899432b611"}
Jul 22 13:23:01.518146 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.518116    2563 scope.go:117] "RemoveContainer" containerID="1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68"
Jul 22 13:23:01.527110 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.527090    2563 scope.go:117] "RemoveContainer" containerID="1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68"
Jul 22 13:23:01.527362 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:23:01.527345    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68\": container with ID starting with 1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68 not found: ID does not exist" containerID="1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68"
Jul 22 13:23:01.527432 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.527368    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68"} err="failed to get container status \"1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68\": rpc error: code = NotFound desc = could not find container \"1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68\": container with ID starting with 1ff777b03de0be9f4868bf60b96e87550e1a280116ec11d363ac83dcf53cdd68 not found: ID does not exist"
Jul 22 13:23:01.540974 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.540949    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"]
Jul 22 13:23:01.543538 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:01.543520    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/sub-test-model-494b12-kserve-686bcf8d75-tnjfh"]
Jul 22 13:23:02.019131 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:02.019097    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="778937c2-1c55-4007-a43e-e859ee103e27" path="/var/lib/kubelet/pods/778937c2-1c55-4007-a43e-e859ee103e27/volumes"
Jul 22 13:23:02.019547 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:02.019532    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0a95514-9ac3-49b9-8f4b-92bc237ae995" path="/var/lib/kubelet/pods/a0a95514-9ac3-49b9-8f4b-92bc237ae995/volumes"
Jul 22 13:23:04.010005 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.009972    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"]
Jul 22 13:23:04.010651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.010618    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="778937c2-1c55-4007-a43e-e859ee103e27" containerName="main"
Jul 22 13:23:04.010651 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.010647    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="778937c2-1c55-4007-a43e-e859ee103e27" containerName="main"
Jul 22 13:23:04.010822 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.010680    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0a95514-9ac3-49b9-8f4b-92bc237ae995" containerName="main"
Jul 22 13:23:04.010822 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.010688    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0a95514-9ac3-49b9-8f4b-92bc237ae995" containerName="main"
Jul 22 13:23:04.010822 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.010799    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0a95514-9ac3-49b9-8f4b-92bc237ae995" containerName="main"
Jul 22 13:23:04.010822 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.010810    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="778937c2-1c55-4007-a43e-e859ee103e27" containerName="main"
Jul 22 13:23:04.013775 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.013734    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.017486 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.017464    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"kube-root-ca.crt\""
Jul 22 13:23:04.017599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.017532    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"default-dockercfg-t45sl\""
Jul 22 13:23:04.017599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.017513    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"openshift-service-ca.crt\""
Jul 22 13:23:04.017599 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.017591    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-634440\"/\"rate-test-model-634440-kserve-self-signed-certs\""
Jul 22 13:23:04.022975 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.022485    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"]
Jul 22 13:23:04.069042 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.069007    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwzq6\" (UniqueName: \"kubernetes.io/projected/36b42365-34c4-4c00-ab94-b458eb98663d-kube-api-access-bwzq6\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.069212 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.069053    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b42365-34c4-4c00-ab94-b458eb98663d-tls-certs\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.069212 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.069108    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-model-cache\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.069212 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.069156    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-tmp-dir\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.069212 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.069186    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-dshm\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.069373 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.069326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-home\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.170813 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.170737    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-home\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.170987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.170867    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwzq6\" (UniqueName: \"kubernetes.io/projected/36b42365-34c4-4c00-ab94-b458eb98663d-kube-api-access-bwzq6\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.170987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.170890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b42365-34c4-4c00-ab94-b458eb98663d-tls-certs\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.170987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.170909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-model-cache\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.170987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.170945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-tmp-dir\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.170987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.170973    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-dshm\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.171267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.171184    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-home\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.171267 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.171211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-model-cache\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.171349 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.171262    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-tmp-dir\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.173440 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.173416    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-dshm\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.173550 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.173532    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b42365-34c4-4c00-ab94-b458eb98663d-tls-certs\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.186191 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.186166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwzq6\" (UniqueName: \"kubernetes.io/projected/36b42365-34c4-4c00-ab94-b458eb98663d-kube-api-access-bwzq6\") pod \"rate-test-model-634440-kserve-6cb6944c79-kwj9q\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") " pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.325321 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.325233    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:04.463791 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.463736    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"]
Jul 22 13:23:04.466308 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:23:04.466279    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36b42365_34c4_4c00_ab94_b458eb98663d.slice/crio-8018ae0797690269f7e2233ba966945223e30cecbfaff55ece9d1741351c979d WatchSource:0}: Error finding container 8018ae0797690269f7e2233ba966945223e30cecbfaff55ece9d1741351c979d: Status 404 returned error can't find the container with id 8018ae0797690269f7e2233ba966945223e30cecbfaff55ece9d1741351c979d
Jul 22 13:23:04.536719 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.536680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q" event={"ID":"36b42365-34c4-4c00-ab94-b458eb98663d","Type":"ContainerStarted","Data":"4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3"}
Jul 22 13:23:04.536719 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.536723    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q" event={"ID":"36b42365-34c4-4c00-ab94-b458eb98663d","Type":"ContainerStarted","Data":"8018ae0797690269f7e2233ba966945223e30cecbfaff55ece9d1741351c979d"}
Jul 22 13:23:04.556247 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:04.556198    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q" podStartSLOduration=1.556181004 podStartE2EDuration="1.556181004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:23:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:23:04.55388871 +0000 UTC m=+2769.018786759" watchObservedRunningTime="2026-07-22 13:23:04.556181004 +0000 UTC m=+2769.021079097"
Jul 22 13:23:14.325786 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:14.325726    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:14.326276 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:14.325801    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:14.338451 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:14.338426    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:14.594824 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:14.594728    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:23:41.610760 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.610709    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"]
Jul 22 13:23:41.614559 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.614539    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.617365 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.617341    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"kube-root-ca.crt\""
Jul 22 13:23:41.618377 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.618355    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"openshift-service-ca.crt\""
Jul 22 13:23:41.618494 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.618429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"default-dockercfg-l9khj\""
Jul 22 13:23:41.618574 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.618560    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-494b12\"/\"rate-test-model-494b12-kserve-self-signed-certs\""
Jul 22 13:23:41.624375 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.624350    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"]
Jul 22 13:23:41.718718 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.718686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a506020-155e-4ed7-866b-a8dd25f83261-tls-certs\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.718937 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.718773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-dshm\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.718937 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.718798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-model-cache\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.718937 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.718832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdbkt\" (UniqueName: \"kubernetes.io/projected/9a506020-155e-4ed7-866b-a8dd25f83261-kube-api-access-sdbkt\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.718937 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.718911    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-home\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.719110 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.718979    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-tmp-dir\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.820589 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.820552    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-home\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821108 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-home\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821220 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821116    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-tmp-dir\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821220 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a506020-155e-4ed7-866b-a8dd25f83261-tls-certs\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821332 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-dshm\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821332 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-model-cache\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821332 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdbkt\" (UniqueName: \"kubernetes.io/projected/9a506020-155e-4ed7-866b-a8dd25f83261-kube-api-access-sdbkt\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821493 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821470    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-tmp-dir\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.821935 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.821913    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-model-cache\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.824034 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.824010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-dshm\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.824299 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.824279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a506020-155e-4ed7-866b-a8dd25f83261-tls-certs\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.830153 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.830125    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdbkt\" (UniqueName: \"kubernetes.io/projected/9a506020-155e-4ed7-866b-a8dd25f83261-kube-api-access-sdbkt\") pod \"rate-test-model-494b12-kserve-664b644db7-t7smd\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") " pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:41.926163 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:41.926076    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:42.069562 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:42.069537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"]
Jul 22 13:23:42.071759 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:23:42.071718    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9a506020_155e_4ed7_866b_a8dd25f83261.slice/crio-ce469488052e37a0c5ee5349a5fab0e22a89edde8acb3952db17e21608a7b8ec WatchSource:0}: Error finding container ce469488052e37a0c5ee5349a5fab0e22a89edde8acb3952db17e21608a7b8ec: Status 404 returned error can't find the container with id ce469488052e37a0c5ee5349a5fab0e22a89edde8acb3952db17e21608a7b8ec
Jul 22 13:23:42.727324 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:42.727286    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd" event={"ID":"9a506020-155e-4ed7-866b-a8dd25f83261","Type":"ContainerStarted","Data":"b52cb223adb833a1ba7ba693c9593f386357b40b3f2cdb2196301f9c1bed5f58"}
Jul 22 13:23:42.727781 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:42.727331    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd" event={"ID":"9a506020-155e-4ed7-866b-a8dd25f83261","Type":"ContainerStarted","Data":"ce469488052e37a0c5ee5349a5fab0e22a89edde8acb3952db17e21608a7b8ec"}
Jul 22 13:23:42.746293 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:42.746249    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd" podStartSLOduration=1.746234486 podStartE2EDuration="1.746234486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 13:23:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:23:42.743680615 +0000 UTC m=+2807.208578663" watchObservedRunningTime="2026-07-22 13:23:42.746234486 +0000 UTC m=+2807.211132535"
Jul 22 13:23:51.926389 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:51.926350    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:51.927003 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:51.926407    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:51.938969 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:51.938902    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:23:52.790084 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:23:52.790046    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:24:53.062918 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:24:53.062835    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"]
Jul 22 13:24:53.996160 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:24:53.996127    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"]
Jul 22 13:25:08.084032 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.083991    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q" podUID="36b42365-34c4-4c00-ab94-b458eb98663d" containerName="main" containerID="cri-o://4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3" gracePeriod=45
Jul 22 13:25:08.330398 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.330374    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:25:08.413154 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413076    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b42365-34c4-4c00-ab94-b458eb98663d-tls-certs\") pod \"36b42365-34c4-4c00-ab94-b458eb98663d\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") "
Jul 22 13:25:08.413154 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413120    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-tmp-dir\") pod \"36b42365-34c4-4c00-ab94-b458eb98663d\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") "
Jul 22 13:25:08.413154 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413145    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-dshm\") pod \"36b42365-34c4-4c00-ab94-b458eb98663d\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") "
Jul 22 13:25:08.413435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413191    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-home\") pod \"36b42365-34c4-4c00-ab94-b458eb98663d\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") "
Jul 22 13:25:08.413435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413212    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-model-cache\") pod \"36b42365-34c4-4c00-ab94-b458eb98663d\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") "
Jul 22 13:25:08.413435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413231    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwzq6\" (UniqueName: \"kubernetes.io/projected/36b42365-34c4-4c00-ab94-b458eb98663d-kube-api-access-bwzq6\") pod \"36b42365-34c4-4c00-ab94-b458eb98663d\" (UID: \"36b42365-34c4-4c00-ab94-b458eb98663d\") "
Jul 22 13:25:08.413435 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413373    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36b42365-34c4-4c00-ab94-b458eb98663d" (UID: "36b42365-34c4-4c00-ab94-b458eb98663d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:08.413639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413451    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-home" (OuterVolumeSpecName: "home") pod "36b42365-34c4-4c00-ab94-b458eb98663d" (UID: "36b42365-34c4-4c00-ab94-b458eb98663d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:08.413639 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413476    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-model-cache" (OuterVolumeSpecName: "model-cache") pod "36b42365-34c4-4c00-ab94-b458eb98663d" (UID: "36b42365-34c4-4c00-ab94-b458eb98663d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:08.413729 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413642    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:08.413729 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413662    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:08.413729 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.413676    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:08.415364 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.415345    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36b42365-34c4-4c00-ab94-b458eb98663d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36b42365-34c4-4c00-ab94-b458eb98663d" (UID: "36b42365-34c4-4c00-ab94-b458eb98663d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:25:08.415433 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.415343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-dshm" (OuterVolumeSpecName: "dshm") pod "36b42365-34c4-4c00-ab94-b458eb98663d" (UID: "36b42365-34c4-4c00-ab94-b458eb98663d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:08.415470 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.415443    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36b42365-34c4-4c00-ab94-b458eb98663d-kube-api-access-bwzq6" (OuterVolumeSpecName: "kube-api-access-bwzq6") pod "36b42365-34c4-4c00-ab94-b458eb98663d" (UID: "36b42365-34c4-4c00-ab94-b458eb98663d"). InnerVolumeSpecName "kube-api-access-bwzq6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:25:08.514868 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.514830    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwzq6\" (UniqueName: \"kubernetes.io/projected/36b42365-34c4-4c00-ab94-b458eb98663d-kube-api-access-bwzq6\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:08.514868 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.514863    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36b42365-34c4-4c00-ab94-b458eb98663d-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:08.515082 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:08.514886    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36b42365-34c4-4c00-ab94-b458eb98663d-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:09.017299 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.017241    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd" podUID="9a506020-155e-4ed7-866b-a8dd25f83261" containerName="main" containerID="cri-o://b52cb223adb833a1ba7ba693c9593f386357b40b3f2cdb2196301f9c1bed5f58" gracePeriod=45
Jul 22 13:25:09.158525 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.158488    2563 generic.go:356] "Generic (PLEG): container finished" podID="36b42365-34c4-4c00-ab94-b458eb98663d" containerID="4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3" exitCode=0
Jul 22 13:25:09.158525 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.158531    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"
Jul 22 13:25:09.159017 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.158575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q" event={"ID":"36b42365-34c4-4c00-ab94-b458eb98663d","Type":"ContainerDied","Data":"4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3"}
Jul 22 13:25:09.159017 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.158619    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q" event={"ID":"36b42365-34c4-4c00-ab94-b458eb98663d","Type":"ContainerDied","Data":"8018ae0797690269f7e2233ba966945223e30cecbfaff55ece9d1741351c979d"}
Jul 22 13:25:09.159017 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.158641    2563 scope.go:117] "RemoveContainer" containerID="4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3"
Jul 22 13:25:09.160524 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.160498    2563 generic.go:356] "Generic (PLEG): container finished" podID="9a506020-155e-4ed7-866b-a8dd25f83261" containerID="b52cb223adb833a1ba7ba693c9593f386357b40b3f2cdb2196301f9c1bed5f58" exitCode=0
Jul 22 13:25:09.160634 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.160576    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd" event={"ID":"9a506020-155e-4ed7-866b-a8dd25f83261","Type":"ContainerDied","Data":"b52cb223adb833a1ba7ba693c9593f386357b40b3f2cdb2196301f9c1bed5f58"}
Jul 22 13:25:09.169223 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.169201    2563 scope.go:117] "RemoveContainer" containerID="4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3"
Jul 22 13:25:09.169516 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:25:09.169494    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3\": container with ID starting with 4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3 not found: ID does not exist" containerID="4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3"
Jul 22 13:25:09.169615 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.169530    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3"} err="failed to get container status \"4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3\": rpc error: code = NotFound desc = could not find container \"4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3\": container with ID starting with 4089c8f40b2060e6c76e360ca1c40091e40eba1136b57dc23deb5f649bc7dca3 not found: ID does not exist"
Jul 22 13:25:09.183359 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.183334    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"]
Jul 22 13:25:09.188837 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.188814    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-634440/rate-test-model-634440-kserve-6cb6944c79-kwj9q"]
Jul 22 13:25:09.282795 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.282706    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:25:09.322058 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322030    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdbkt\" (UniqueName: \"kubernetes.io/projected/9a506020-155e-4ed7-866b-a8dd25f83261-kube-api-access-sdbkt\") pod \"9a506020-155e-4ed7-866b-a8dd25f83261\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") "
Jul 22 13:25:09.322225 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322069    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-dshm\") pod \"9a506020-155e-4ed7-866b-a8dd25f83261\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") "
Jul 22 13:25:09.322225 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322088    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-home\") pod \"9a506020-155e-4ed7-866b-a8dd25f83261\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") "
Jul 22 13:25:09.322225 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322143    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-tmp-dir\") pod \"9a506020-155e-4ed7-866b-a8dd25f83261\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") "
Jul 22 13:25:09.322225 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322174    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a506020-155e-4ed7-866b-a8dd25f83261-tls-certs\") pod \"9a506020-155e-4ed7-866b-a8dd25f83261\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") "
Jul 22 13:25:09.322442 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322237    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-model-cache\") pod \"9a506020-155e-4ed7-866b-a8dd25f83261\" (UID: \"9a506020-155e-4ed7-866b-a8dd25f83261\") "
Jul 22 13:25:09.322502 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322445    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9a506020-155e-4ed7-866b-a8dd25f83261" (UID: "9a506020-155e-4ed7-866b-a8dd25f83261"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:09.322502 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-home" (OuterVolumeSpecName: "home") pod "9a506020-155e-4ed7-866b-a8dd25f83261" (UID: "9a506020-155e-4ed7-866b-a8dd25f83261"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:09.322616 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322588    2563 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-home\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:09.322616 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322607    2563 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-tmp-dir\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:09.322705 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.322642    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-model-cache" (OuterVolumeSpecName: "model-cache") pod "9a506020-155e-4ed7-866b-a8dd25f83261" (UID: "9a506020-155e-4ed7-866b-a8dd25f83261"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:09.324359 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.324338    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-dshm" (OuterVolumeSpecName: "dshm") pod "9a506020-155e-4ed7-866b-a8dd25f83261" (UID: "9a506020-155e-4ed7-866b-a8dd25f83261"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:09.324452 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.324422    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a506020-155e-4ed7-866b-a8dd25f83261-kube-api-access-sdbkt" (OuterVolumeSpecName: "kube-api-access-sdbkt") pod "9a506020-155e-4ed7-866b-a8dd25f83261" (UID: "9a506020-155e-4ed7-866b-a8dd25f83261"). InnerVolumeSpecName "kube-api-access-sdbkt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:25:09.324794 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.324739    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a506020-155e-4ed7-866b-a8dd25f83261-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9a506020-155e-4ed7-866b-a8dd25f83261" (UID: "9a506020-155e-4ed7-866b-a8dd25f83261"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 22 13:25:09.423600 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.423563    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdbkt\" (UniqueName: \"kubernetes.io/projected/9a506020-155e-4ed7-866b-a8dd25f83261-kube-api-access-sdbkt\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:09.423600 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.423596    2563 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-dshm\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:09.423600 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.423608    2563 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9a506020-155e-4ed7-866b-a8dd25f83261-tls-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:09.423882 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:09.423618    2563 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9a506020-155e-4ed7-866b-a8dd25f83261-model-cache\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:10.019145 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:10.019115    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36b42365-34c4-4c00-ab94-b458eb98663d" path="/var/lib/kubelet/pods/36b42365-34c4-4c00-ab94-b458eb98663d/volumes"
Jul 22 13:25:10.168377 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:10.168342    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd" event={"ID":"9a506020-155e-4ed7-866b-a8dd25f83261","Type":"ContainerDied","Data":"ce469488052e37a0c5ee5349a5fab0e22a89edde8acb3952db17e21608a7b8ec"}
Jul 22 13:25:10.168880 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:10.168384    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"
Jul 22 13:25:10.168880 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:10.168400    2563 scope.go:117] "RemoveContainer" containerID="b52cb223adb833a1ba7ba693c9593f386357b40b3f2cdb2196301f9c1bed5f58"
Jul 22 13:25:10.187911 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:10.187884    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"]
Jul 22 13:25:10.195833 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:10.195809    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-494b12/rate-test-model-494b12-kserve-664b644db7-t7smd"]
Jul 22 13:25:12.018383 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:12.018343    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a506020-155e-4ed7-866b-a8dd25f83261" path="/var/lib/kubelet/pods/9a506020-155e-4ed7-866b-a8dd25f83261/volumes"
Jul 22 13:25:36.420325 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:36.420291    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"]
Jul 22 13:25:36.420825 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:36.420481    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy" containerID="cri-o://8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc" gracePeriod=30
Jul 22 13:25:37.985706 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:37.985669    2563 patch_prober.go:28] interesting pod/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 22 13:25:37.986116 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:37.985731    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 22 13:25:41.668760 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.668724    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:25:41.710716 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.710691    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-socket\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.710893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.710772    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05794723-c5b4-41e2-bc62-191dfa59771f-istiod-ca-cert\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.710893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.710793    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-envoy\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.710893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.710823    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-credential-socket\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.710893 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.710852    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05794723-c5b4-41e2-bc62-191dfa59771f-istio-podinfo\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.711182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.711023    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:41.711182 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.711160    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:41.711278 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.711188    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/05794723-c5b4-41e2-bc62-191dfa59771f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:25:41.713092 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.713062    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:41.713191 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.713160    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/05794723-c5b4-41e2-bc62-191dfa59771f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:25:41.811448 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811420    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-istio-token\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.811615 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811455    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-data\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.811615 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811481    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k8nk5\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-kube-api-access-k8nk5\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.811615 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811497    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-certs\") pod \"05794723-c5b4-41e2-bc62-191dfa59771f\" (UID: \"05794723-c5b4-41e2-bc62-191dfa59771f\") "
Jul 22 13:25:41.811830 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811633    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/05794723-c5b4-41e2-bc62-191dfa59771f-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.811830 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811645    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.811830 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811653    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.811830 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811663    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/05794723-c5b4-41e2-bc62-191dfa59771f-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.811830 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811672    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.811830 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.811809    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:41.812343 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.812318    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-data" (OuterVolumeSpecName: "istio-data") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:25:41.813566 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.813548    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-istio-token" (OuterVolumeSpecName: "istio-token") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:25:41.813617 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.813589    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-kube-api-access-k8nk5" (OuterVolumeSpecName: "kube-api-access-k8nk5") pod "05794723-c5b4-41e2-bc62-191dfa59771f" (UID: "05794723-c5b4-41e2-bc62-191dfa59771f"). InnerVolumeSpecName "kube-api-access-k8nk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:25:41.912363 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.912337    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.912363 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.912361    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.912610 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.912371    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k8nk5\" (UniqueName: \"kubernetes.io/projected/05794723-c5b4-41e2-bc62-191dfa59771f-kube-api-access-k8nk5\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:41.912610 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:41.912383    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/05794723-c5b4-41e2-bc62-191dfa59771f-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:25:42.333764 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.333661    2563 generic.go:356] "Generic (PLEG): container finished" podID="05794723-c5b4-41e2-bc62-191dfa59771f" containerID="8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc" exitCode=0
Jul 22 13:25:42.333764 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.333704    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"
Jul 22 13:25:42.333764 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.333709    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" event={"ID":"05794723-c5b4-41e2-bc62-191dfa59771f","Type":"ContainerDied","Data":"8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc"}
Jul 22 13:25:42.333990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.333775    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr" event={"ID":"05794723-c5b4-41e2-bc62-191dfa59771f","Type":"ContainerDied","Data":"b4d60e2b8a4571c347d81035ea46f14c1b596e57efdbf4df45e0aed783ca2c0e"}
Jul 22 13:25:42.333990 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.333802    2563 scope.go:117] "RemoveContainer" containerID="8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc"
Jul 22 13:25:42.342783 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.342763    2563 scope.go:117] "RemoveContainer" containerID="f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5"
Jul 22 13:25:42.350603 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.350587    2563 scope.go:117] "RemoveContainer" containerID="8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc"
Jul 22 13:25:42.350873 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:25:42.350855    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc\": container with ID starting with 8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc not found: ID does not exist" containerID="8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc"
Jul 22 13:25:42.350934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.350880    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc"} err="failed to get container status \"8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc\": rpc error: code = NotFound desc = could not find container \"8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc\": container with ID starting with 8337aa93b5d5f8ba6b78909862788d12991c611040d305fb503c8ebfd12b42cc not found: ID does not exist"
Jul 22 13:25:42.350934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.350896    2563 scope.go:117] "RemoveContainer" containerID="f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5"
Jul 22 13:25:42.351183 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:25:42.351165    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5\": container with ID starting with f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5 not found: ID does not exist" containerID="f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5"
Jul 22 13:25:42.351237 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.351189    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5"} err="failed to get container status \"f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5\": rpc error: code = NotFound desc = could not find container \"f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5\": container with ID starting with f4a4d4381fae08ae671c64a8085d62f974d020366b8bebe3b15a7a0b8f435cf5 not found: ID does not exist"
Jul 22 13:25:42.357652 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.357620    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"]
Jul 22 13:25:42.360667 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:42.360648    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-634440-openshift-default-69f76cf6bc-ktnjr"]
Jul 22 13:25:44.018565 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:44.018531    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" path="/var/lib/kubelet/pods/05794723-c5b4-41e2-bc62-191dfa59771f/volumes"
Jul 22 13:25:58.735730 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:58.735696    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"]
Jul 22 13:25:58.736260 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:25:58.735932    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy" containerID="cri-o://3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87" gracePeriod=30
Jul 22 13:26:03.999833 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:03.999772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:26:04.097084 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097046    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-socket\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097084 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097097    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pgqhz\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-kube-api-access-pgqhz\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097126    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9067401-0f27-4bdc-91df-8d773093fdb2-istiod-ca-cert\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097164    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-envoy\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097224    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-data\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097246    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-credential-socket\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097287    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-token\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097299    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:26:04.097330 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097317    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-certs\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097689 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-podinfo\") pod \"c9067401-0f27-4bdc-91df-8d773093fdb2\" (UID: \"c9067401-0f27-4bdc-91df-8d773093fdb2\") "
Jul 22 13:26:04.097689 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097546    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:26:04.097689 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097572    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:26:04.097689 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097609    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9067401-0f27-4bdc-91df-8d773093fdb2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 22 13:26:04.097910 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097845    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-credential-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.097992 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.097894    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-certs\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.098059 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.098005    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-workload-socket\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.098116 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.098061    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c9067401-0f27-4bdc-91df-8d773093fdb2-istiod-ca-cert\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.098175 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.098152    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-data" (OuterVolumeSpecName: "istio-data") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:26:04.099659 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.099627    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-kube-api-access-pgqhz" (OuterVolumeSpecName: "kube-api-access-pgqhz") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "kube-api-access-pgqhz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:26:04.099825 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.099683    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 22 13:26:04.099825 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.099730    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 22 13:26:04.099987 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.099966    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-token" (OuterVolumeSpecName: "istio-token") pod "c9067401-0f27-4bdc-91df-8d773093fdb2" (UID: "c9067401-0f27-4bdc-91df-8d773093fdb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 22 13:26:04.198879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.198830    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-envoy\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.198879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.198873    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-data\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.198879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.198882    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-token\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.198879 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.198890    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c9067401-0f27-4bdc-91df-8d773093fdb2-istio-podinfo\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.199158 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.198900    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pgqhz\" (UniqueName: \"kubernetes.io/projected/c9067401-0f27-4bdc-91df-8d773093fdb2-kube-api-access-pgqhz\") on node \"ip-10-0-128-197.ec2.internal\" DevicePath \"\""
Jul 22 13:26:04.443349 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.443317    2563 generic.go:356] "Generic (PLEG): container finished" podID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerID="3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87" exitCode=0
Jul 22 13:26:04.443559 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.443373    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"
Jul 22 13:26:04.443559 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.443409    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" event={"ID":"c9067401-0f27-4bdc-91df-8d773093fdb2","Type":"ContainerDied","Data":"3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87"}
Jul 22 13:26:04.443559 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.443462    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685" event={"ID":"c9067401-0f27-4bdc-91df-8d773093fdb2","Type":"ContainerDied","Data":"f5e6acd9c92623b52ec3480c8666f484c0a03a2d1aeeee2f1852d4639bbbdea7"}
Jul 22 13:26:04.443559 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.443484    2563 scope.go:117] "RemoveContainer" containerID="3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87"
Jul 22 13:26:04.453086 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.453070    2563 scope.go:117] "RemoveContainer" containerID="1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13"
Jul 22 13:26:04.461060 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.461045    2563 scope.go:117] "RemoveContainer" containerID="3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87"
Jul 22 13:26:04.461293 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:26:04.461274    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87\": container with ID starting with 3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87 not found: ID does not exist" containerID="3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87"
Jul 22 13:26:04.461346 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.461301    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87"} err="failed to get container status \"3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87\": rpc error: code = NotFound desc = could not find container \"3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87\": container with ID starting with 3fe10d512f45a2a7c7b63ec09513252763a07f705b78814d90e503422c380a87 not found: ID does not exist"
Jul 22 13:26:04.461346 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.461318    2563 scope.go:117] "RemoveContainer" containerID="1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13"
Jul 22 13:26:04.461583 ip-10-0-128-197 kubenswrapper[2563]: E0722 13:26:04.461566    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13\": container with ID starting with 1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13 not found: ID does not exist" containerID="1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13"
Jul 22 13:26:04.461628 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.461590    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13"} err="failed to get container status \"1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13\": rpc error: code = NotFound desc = could not find container \"1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13\": container with ID starting with 1e05967ae7be670b228699e8bdeccd089df72d7b3f1124411086bc9911983a13 not found: ID does not exist"
Jul 22 13:26:04.465389 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.465369    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"]
Jul 22 13:26:04.469290 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.469268    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-494b12-openshift-default-5fd9869b59-9h685"]
Jul 22 13:26:04.960760 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:04.960718    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-k2qvn_400cffb6-63ed-4f65-96b8-ad1dfbdfea02/manager/0.log"
Jul 22 13:26:05.091424 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:05.091386    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-s2kl7_f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9/manager/0.log"
Jul 22 13:26:05.385016 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:05.384941    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-krbjc_a77a4148-91cc-4838-bf15-d2dfa9357c7d/server/0.log"
Jul 22 13:26:05.508860 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:05.508833    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ct2g7_8d2eb7f9-faea-44dd-8f39-693e5bb0fe64/manager/0.log"
Jul 22 13:26:05.905857 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:05.905833    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99b87cc7-s9h9f_f2c6eb62-befc-4117-a0ed-5cd14043d9f0/copy-manifests/0.log"
Jul 22 13:26:05.916014 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:05.915982    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99b87cc7-s9h9f_f2c6eb62-befc-4117-a0ed-5cd14043d9f0/manager/0.log"
Jul 22 13:26:06.018121 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:06.018090    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" path="/var/lib/kubelet/pods/c9067401-0f27-4bdc-91df-8d773093fdb2/volumes"
Jul 22 13:26:06.278729 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:06.278685    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-684f74c5d4-7qpm8_10a2b508-9335-4069-ad7f-851dcd000389/maas-api/0.log"
Jul 22 13:26:06.626062 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:06.625981    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-ljwq7_ce0de7c3-21db-4b3c-8612-62857d1130a3/postgres/0.log"
Jul 22 13:26:08.078505 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:08.078472    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mqp5k_3277f9b3-452f-4185-847f-5a8fb6e1aec5/manager/0.log"
Jul 22 13:26:08.195503 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:08.195474    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8fhxk_ab974f39-7b3f-43e3-9f09-b1bce0d001c1/manager/0.log"
Jul 22 13:26:08.439968 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:08.439929    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-77w8d_28425d84-d69e-4b45-b430-541441021e0d/registry-server/0.log"
Jul 22 13:26:08.804566 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:08.804462    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mnkcr_51bf01fa-5516-42ee-b381-3cea360fd927/manager/0.log"
Jul 22 13:26:09.177163 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:09.177082    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xx2dp_b89650cf-0d00-4bd4-b26f-5d1316817e5c/istio-proxy/0.log"
Jul 22 13:26:09.426999 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:09.426970    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d54956c5-6296t_74ed1838-9d65-46a7-bfd3-cfafd39861cd/kube-auth-proxy/0.log"
Jul 22 13:26:09.667018 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:09.666986    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4jkgx_d664d98d-27de-4f92-ac66-bc32e06d6209/istio-proxy/1.log"
Jul 22 13:26:09.789504 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:09.789463    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pdkgm_c68873a9-6030-4711-9caf-b29cece27106/payload-pre-processing/0.log"
Jul 22 13:26:09.915300 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:09.915257    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-gxkfc_91c7f6e1-c5f9-4e88-9aed-3a529646e1c4/payload-processing/0.log"
Jul 22 13:26:10.515144 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:10.515112    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-v5z9g_f7e87297-7204-43d7-9839-1171ee1092ce/main/0.log"
Jul 22 13:26:10.875708 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:10.875621    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-2vm7k_8a785aec-ec3a-4e10-b41b-8d31b77d6227/main/0.log"
Jul 22 13:26:10.990994 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:10.990965    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-bn8q8_238c37cf-948a-4c13-8880-6cd2f0eb1c15/main/0.log"
Jul 22 13:26:20.118519 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:20.118437    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zfw59_9185d527-afa2-47e7-ac1b-73aae2dcbc9c/global-pull-secret-syncer/0.log"
Jul 22 13:26:20.264829 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:20.264760    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qxt4l_657fc080-2e63-4659-8391-a624921d6a7b/konnectivity-agent/0.log"
Jul 22 13:26:20.289221 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:20.289186    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-197.ec2.internal_f81c43c62f71f9af9b24ee2595804b9b/haproxy/0.log"
Jul 22 13:26:25.454555 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:25.454526    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mqp5k_3277f9b3-452f-4185-847f-5a8fb6e1aec5/manager/0.log"
Jul 22 13:26:25.481486 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:25.481454    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-8fhxk_ab974f39-7b3f-43e3-9f09-b1bce0d001c1/manager/0.log"
Jul 22 13:26:25.568779 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:25.568728    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-77w8d_28425d84-d69e-4b45-b430-541441021e0d/registry-server/0.log"
Jul 22 13:26:25.901934 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:25.901809    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-mnkcr_51bf01fa-5516-42ee-b381-3cea360fd927/manager/0.log"
Jul 22 13:26:28.051978 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:28.051945    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lf8hv_61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f/node-exporter/0.log"
Jul 22 13:26:28.076376 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:28.076348    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lf8hv_61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f/kube-rbac-proxy/0.log"
Jul 22 13:26:28.099721 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:28.099700    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lf8hv_61d2b0d5-a1dd-4475-b642-a3f2f62b1d8f/init-textfile/0.log"
Jul 22 13:26:30.260434 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260403    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"]
Jul 22 13:26:30.260859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260793    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy"
Jul 22 13:26:30.260859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260811    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy"
Jul 22 13:26:30.260859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260824    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy"
Jul 22 13:26:30.260859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260830    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy"
Jul 22 13:26:30.260859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260848    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a506020-155e-4ed7-866b-a8dd25f83261" containerName="main"
Jul 22 13:26:30.260859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260854    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a506020-155e-4ed7-866b-a8dd25f83261" containerName="main"
Jul 22 13:26:30.260859 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260863    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260869    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260883    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36b42365-34c4-4c00-ab94-b458eb98663d" containerName="main"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260891    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="36b42365-34c4-4c00-ab94-b458eb98663d" containerName="main"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260947    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260954    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="05794723-c5b4-41e2-bc62-191dfa59771f" containerName="istio-proxy"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260960    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260968    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a506020-155e-4ed7-866b-a8dd25f83261" containerName="main"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260977    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="36b42365-34c4-4c00-ab94-b458eb98663d" containerName="main"
Jul 22 13:26:30.261173 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.260985    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9067401-0f27-4bdc-91df-8d773093fdb2" containerName="istio-proxy"
Jul 22 13:26:30.265543 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.265526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.268291 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.268270    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wsp6t\"/\"openshift-service-ca.crt\""
Jul 22 13:26:30.268421 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.268400    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-wsp6t\"/\"default-dockercfg-hpnfs\""
Jul 22 13:26:30.268658 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.268645    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-wsp6t\"/\"kube-root-ca.crt\""
Jul 22 13:26:30.274692 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.274668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"]
Jul 22 13:26:30.425327 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.425294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-proc\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.425503 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.425348    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpfxx\" (UniqueName: \"kubernetes.io/projected/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-kube-api-access-zpfxx\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.425503 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.425383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-podres\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.425503 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.425406    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-lib-modules\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.425503 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.425433    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-sys\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.526691 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526597    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-proc\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.526691 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526649    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpfxx\" (UniqueName: \"kubernetes.io/projected/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-kube-api-access-zpfxx\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.526691 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-podres\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.526691 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526694    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-lib-modules\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.527018 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526719    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-sys\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.527018 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526741    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-proc\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.527018 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-sys\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.527018 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526867    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-podres\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.527018 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.526878    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-lib-modules\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.535144 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.535123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpfxx\" (UniqueName: \"kubernetes.io/projected/fb791cbe-5fcf-4eff-b2f2-014dca06b6df-kube-api-access-zpfxx\") pod \"perf-node-gather-daemonset-ttc7n\" (UID: \"fb791cbe-5fcf-4eff-b2f2-014dca06b6df\") " pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.576551 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.576531    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:30.711551 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:30.711529    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"]
Jul 22 13:26:30.713715 ip-10-0-128-197 kubenswrapper[2563]: W0722 13:26:30.713682    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podfb791cbe_5fcf_4eff_b2f2_014dca06b6df.slice/crio-5341d6314aacbbbca26b071328dbb68088a1c6e07083106f54d75a6619d04a81 WatchSource:0}: Error finding container 5341d6314aacbbbca26b071328dbb68088a1c6e07083106f54d75a6619d04a81: Status 404 returned error can't find the container with id 5341d6314aacbbbca26b071328dbb68088a1c6e07083106f54d75a6619d04a81
Jul 22 13:26:31.572226 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:31.572188    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n" event={"ID":"fb791cbe-5fcf-4eff-b2f2-014dca06b6df","Type":"ContainerStarted","Data":"fe1964d6d2d9370f4ba44ad0d9ddf8569c5c2e008f78e40dcfcc9044e80dc364"}
Jul 22 13:26:31.572641 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:31.572344    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:31.572641 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:31.572364    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n" event={"ID":"fb791cbe-5fcf-4eff-b2f2-014dca06b6df","Type":"ContainerStarted","Data":"5341d6314aacbbbca26b071328dbb68088a1c6e07083106f54d75a6619d04a81"}
Jul 22 13:26:31.591582 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:31.591536    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n" podStartSLOduration=1.591521868 podStartE2EDuration="1.591521868s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 13:26:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 13:26:31.589811489 +0000 UTC m=+2976.054709538" watchObservedRunningTime="2026-07-22 13:26:31.591521868 +0000 UTC m=+2976.056419917"
Jul 22 13:26:33.153602 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:33.153574    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-spfbs_420f2a0e-48e0-4144-9118-d954a0ccd28a/dns/0.log"
Jul 22 13:26:33.175407 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:33.175384    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-spfbs_420f2a0e-48e0-4144-9118-d954a0ccd28a/kube-rbac-proxy/0.log"
Jul 22 13:26:33.199652 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:33.199624    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-7cb9k_847d1d73-3c3c-4cce-a5ce-2f05752884f5/dns-node-resolver/0.log"
Jul 22 13:26:33.865843 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:33.865811    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-xvbww_dba8b7c7-8dd2-4913-ab86-8e65582cb93f/node-ca/0.log"
Jul 22 13:26:34.840042 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:34.840009    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xx2dp_b89650cf-0d00-4bd4-b26f-5d1316817e5c/istio-proxy/0.log"
Jul 22 13:26:34.991670 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:34.991638    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d54956c5-6296t_74ed1838-9d65-46a7-bfd3-cfafd39861cd/kube-auth-proxy/0.log"
Jul 22 13:26:35.049466 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:35.049436    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4jkgx_d664d98d-27de-4f92-ac66-bc32e06d6209/istio-proxy/0.log"
Jul 22 13:26:35.103358 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:35.103260    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4jkgx_d664d98d-27de-4f92-ac66-bc32e06d6209/istio-proxy/1.log"
Jul 22 13:26:35.147802 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:35.147776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pdkgm_c68873a9-6030-4711-9caf-b29cece27106/payload-pre-processing/0.log"
Jul 22 13:26:35.193766 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:35.193722    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-gxkfc_91c7f6e1-c5f9-4e88-9aed-3a529646e1c4/payload-processing/0.log"
Jul 22 13:26:35.936305 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:35.936278    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-twkmz_5601a01b-0669-445d-8b56-3f843ec62a65/serve-healthcheck-canary/0.log"
Jul 22 13:26:36.478228 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:36.478192    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5s6bp_d9507753-cea0-4d7b-839d-5da83cd24d4d/kube-rbac-proxy/0.log"
Jul 22 13:26:36.501555 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:36.501527    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5s6bp_d9507753-cea0-4d7b-839d-5da83cd24d4d/exporter/0.log"
Jul 22 13:26:36.526989 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:36.526953    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5s6bp_d9507753-cea0-4d7b-839d-5da83cd24d4d/extractor/0.log"
Jul 22 13:26:37.586633 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:37.586606    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-wsp6t/perf-node-gather-daemonset-ttc7n"
Jul 22 13:26:38.894832 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:38.894799    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-k2qvn_400cffb6-63ed-4f65-96b8-ad1dfbdfea02/manager/0.log"
Jul 22 13:26:39.240529 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:39.240490    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-s2kl7_f5f2df0b-dc00-4193-8dc1-8eda4bec4ea9/manager/0.log"
Jul 22 13:26:39.379527 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:39.379497    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-krbjc_a77a4148-91cc-4838-bf15-d2dfa9357c7d/server/0.log"
Jul 22 13:26:39.501382 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:39.501297    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-ct2g7_8d2eb7f9-faea-44dd-8f39-693e5bb0fe64/manager/0.log"
Jul 22 13:26:39.740611 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:39.740580    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99b87cc7-s9h9f_f2c6eb62-befc-4117-a0ed-5cd14043d9f0/manager/0.log"
Jul 22 13:26:39.769445 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:39.769378    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d99b87cc7-s9h9f_f2c6eb62-befc-4117-a0ed-5cd14043d9f0/copy-manifests/0.log"
Jul 22 13:26:40.686569 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:40.686481    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xx2dp_b89650cf-0d00-4bd4-b26f-5d1316817e5c/istio-proxy/0.log"
Jul 22 13:26:40.826680 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:40.826651    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d54956c5-6296t_74ed1838-9d65-46a7-bfd3-cfafd39861cd/kube-auth-proxy/0.log"
Jul 22 13:26:40.869027 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:40.868999    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4jkgx_d664d98d-27de-4f92-ac66-bc32e06d6209/istio-proxy/0.log"
Jul 22 13:26:40.920695 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:40.920667    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4jkgx_d664d98d-27de-4f92-ac66-bc32e06d6209/istio-proxy/1.log"
Jul 22 13:26:40.957770 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:40.957717    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pdkgm_c68873a9-6030-4711-9caf-b29cece27106/payload-pre-processing/0.log"
Jul 22 13:26:40.988548 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:40.988523    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-gxkfc_91c7f6e1-c5f9-4e88-9aed-3a529646e1c4/payload-processing/0.log"
Jul 22 13:26:41.092376 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:41.092332    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-587h5_41cc80f4-6019-4efd-8352-7b29cfe4d4f8/openshift-lws-operator/0.log"
Jul 22 13:26:42.225733 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:42.225703    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4xx2dp_b89650cf-0d00-4bd4-b26f-5d1316817e5c/istio-proxy/0.log"
Jul 22 13:26:42.371331 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:42.371251    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77d54956c5-6296t_74ed1838-9d65-46a7-bfd3-cfafd39861cd/kube-auth-proxy/0.log"
Jul 22 13:26:42.409569 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:42.409531    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4jkgx_d664d98d-27de-4f92-ac66-bc32e06d6209/istio-proxy/0.log"
Jul 22 13:26:42.462696 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:42.462666    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-4jkgx_d664d98d-27de-4f92-ac66-bc32e06d6209/istio-proxy/1.log"
Jul 22 13:26:42.493918 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:42.493891    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-pdkgm_c68873a9-6030-4711-9caf-b29cece27106/payload-pre-processing/0.log"
Jul 22 13:26:42.522805 ip-10-0-128-197 kubenswrapper[2563]: I0722 13:26:42.522776    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-gxkfc_91c7f6e1-c5f9-4e88-9aed-3a529646e1c4/payload-processing/0.log"