Jul 06 18:57:02.237541 ip-10-0-142-165 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 06 18:57:02.237552 ip-10-0-142-165 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 06 18:57:02.237560 ip-10-0-142-165 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 06 18:57:02.237854 ip-10-0-142-165 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 06 18:57:12.404061 ip-10-0-142-165 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 06 18:57:12.404077 ip-10-0-142-165 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 384c7e3e8afd499fbb6808f2b9f16b95 --
Jul 06 18:59:00.265928 ip-10-0-142-165 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.697272    2579 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701923    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701933    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701937    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701942    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701946    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701950    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701953    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:59:00.800238 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701956    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:59:00.778812 ip-10-0-142-165 systemd[1]: Started Kubernetes Kubelet.
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701959    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701963    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701966    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701970    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701973    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701975    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701978    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701980    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701983    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701986    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701988    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701991    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701993    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701996    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.701999    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702002    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702004    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702007    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702010    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:59:00.801871 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702013    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702023    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702026    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702028    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702031    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702034    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702037    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702040    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702042    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702045    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702048    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702050    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702053    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702055    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702058    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702060    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702063    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702065    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702068    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702070    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:59:00.804376 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702073    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702075    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702078    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702080    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702083    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702085    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702088    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702090    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702093    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702095    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702097    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702100    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702102    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702105    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702107    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702110    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702112    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702115    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702117    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702120    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:59:00.805436 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702123    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702126    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702128    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702130    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702133    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702135    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702138    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702140    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702143    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702145    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702148    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702150    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702154    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702156    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702159    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702161    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702164    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702166    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702169    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:59:00.806562 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702171    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702174    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702176    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702179    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702182    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702184    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702187    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702190    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702193    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702195    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702199    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702203    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702207    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702888    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702894    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702897    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702900    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702903    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702907    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:59:00.807388 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702910    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702912    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702915    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702918    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702921    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702923    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702926    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702929    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702932    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702934    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702937    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702940    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702943    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702946    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702949    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702951    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702953    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702956    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702959    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702961    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:59:00.808035 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702963    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702966    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702968    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702971    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702973    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702976    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702979    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702981    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702984    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702986    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702988    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702991    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702994    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702996    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.702999    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703001    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703003    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703006    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703009    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703011    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:59:00.809074 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703014    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703016    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703019    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703022    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703024    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703026    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703029    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703032    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703037    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703040    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703043    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703046    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703049    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703052    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703054    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703056    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703059    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703061    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703064    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:59:00.809927 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703066    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703069    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703072    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703074    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703077    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703079    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703082    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703084    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703087    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703089    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703092    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703097    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703100    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703103    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703106    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703109    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703112    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703114    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703117    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703119    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:59:00.810546 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703122    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703124    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703128    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703132    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703135    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703137    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703140    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703143    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703146    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703148    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703151    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703153    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703156    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703158    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703498    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703503    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703506    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703508    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703511    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703514    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:59:00.811190 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703516    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703519    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703523    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703526    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703529    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703531    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703535    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703537    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703540    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703542    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703545    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703548    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703550    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703553    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703555    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703558    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703561    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703563    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703565    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703568    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703570    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:59:00.811803 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703573    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703576    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703578    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703580    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703583    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703585    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703588    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703590    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703593    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703595    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703597    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703601    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703605    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703609    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703613    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703617    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703620    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703624    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703627    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:59:00.812417 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703630    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703633    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703635    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703638    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703640    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703643    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703645    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703648    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703650    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703668    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703670    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703673    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703675    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703678    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703681    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703683    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703686    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703688    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703691    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703694    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:59:00.813023 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703696    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703698    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703701    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703703    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703706    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703708    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703712    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703715    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703719    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703723    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703726    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703729    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703733    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703736    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703738    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703741    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703743    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703746    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703748    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:59:00.813626 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703751    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703753    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703756    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703758    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703761    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703763    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703766    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703768    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703771    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703773    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703776    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703778    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703781    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.703783    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703847    2579 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703853    2579 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703859    2579 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703863    2579 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703867    2579 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703871    2579 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703874    2579 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 18:59:00.814159 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703880    2579 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703883    2579 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703886    2579 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703889    2579 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703892    2579 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703896    2579 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703899    2579 flags.go:64] FLAG: --cgroup-root=""
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703902    2579 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703905    2579 flags.go:64] FLAG: --client-ca-file=""
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703908    2579 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703911    2579 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703914    2579 flags.go:64] FLAG: --cluster-domain=""
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703917    2579 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703920    2579 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703923    2579 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703927    2579 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703930    2579 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703932    2579 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703935    2579 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703939    2579 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703941    2579 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703944    2579 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703947    2579 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703950    2579 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 18:59:00.814928 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703953    2579 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703956    2579 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703959    2579 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703962    2579 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703965    2579 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703967    2579 flags.go:64] FLAG: --enable-server="true"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703972    2579 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703976    2579 flags.go:64] FLAG: --event-burst="100"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703980    2579 flags.go:64] FLAG: --event-qps="50"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703983    2579 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703986    2579 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703989    2579 flags.go:64] FLAG: --eviction-hard=""
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703992    2579 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703995    2579 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.703998    2579 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704001    2579 flags.go:64] FLAG: --eviction-soft=""
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704004    2579 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704007    2579 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704009    2579 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704012    2579 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704015    2579 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704017    2579 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704020    2579 flags.go:64] FLAG: --feature-gates=""
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704024    2579 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704026    2579 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 18:59:00.815647 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704029    2579 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704032    2579 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704035    2579 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704038    2579 flags.go:64] FLAG: --help="false"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704040    2579 flags.go:64] FLAG: --hostname-override="ip-10-0-142-165.ec2.internal"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704043    2579 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704046    2579 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704049    2579 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704052    2579 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704055    2579 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704058    2579 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704061    2579 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704064    2579 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704068    2579 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704085    2579 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704090    2579 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704093    2579 flags.go:64] FLAG: --kube-reserved=""
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704096    2579 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704099    2579 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704102    2579 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704105    2579 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704108    2579 flags.go:64] FLAG: --lock-file=""
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704111    2579 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704114    2579 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 18:59:00.816416 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704117    2579 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704122    2579 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704124    2579 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704127    2579 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704130    2579 flags.go:64] FLAG: --logging-format="text"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704132    2579 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704136    2579 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704138    2579 flags.go:64] FLAG: --manifest-url=""
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704141    2579 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704145    2579 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704148    2579 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704151    2579 flags.go:64] FLAG: --max-pods="110"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704154    2579 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704157    2579 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704160    2579 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704162    2579 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704165    2579 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704168    2579 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704171    2579 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704177    2579 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704180    2579 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704184    2579 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704187    2579 flags.go:64] FLAG: --pod-cidr=""
Jul 06 18:59:00.817367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704191    2579 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704196    2579 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704199    2579 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704202    2579 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704204    2579 flags.go:64] FLAG: --port="10250"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704207    2579 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704210    2579 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-011ec50181ed8f8f3"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704213    2579 flags.go:64] FLAG: --qos-reserved=""
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704216    2579 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704219    2579 flags.go:64] FLAG: --register-node="true"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704222    2579 flags.go:64] FLAG: --register-with-taints=""
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704225    2579 flags.go:64] FLAG: --registry-burst="10"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704228    2579 flags.go:64] FLAG: --registry-qps="5"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704231    2579 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704233    2579 flags.go:64] FLAG: --reserved-memory=""
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704237    2579 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704240    2579 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704243    2579 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704245    2579 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704248    2579 flags.go:64] FLAG: --runonce="false"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704251    2579 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704253    2579 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704256    2579 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704259    2579 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704262    2579 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704264    2579 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 18:59:00.941076 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704267    2579 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704270    2579 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704273    2579 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704276    2579 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704279    2579 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704282    2579 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704285    2579 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704289    2579 flags.go:64] FLAG: --system-cgroups=""
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704291    2579 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704296    2579 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704299    2579 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704302    2579 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704307    2579 flags.go:64] FLAG: --tls-min-version=""
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704310    2579 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704312    2579 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704316    2579 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704318    2579 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704321    2579 flags.go:64] FLAG: --v="2"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704325    2579 flags.go:64] FLAG: --version="false"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704328    2579 flags.go:64] FLAG: --vmodule=""
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704331    2579 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704335    2579 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704443    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704447    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:59:00.961640 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704450    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704453    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704455    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704458    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704461    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704463    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704466    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704468    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704471    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704474    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704483    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704485    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704489    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704492    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704494    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704498    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704501    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704503    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704505    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704508    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:59:00.962482 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704511    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704513    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704516    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704518    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704530    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704533    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704535    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704538    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704540    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704542    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704545    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704547    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704550    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704552    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704555    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704557    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704560    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704562    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704565    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704567    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:59:00.963171 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704570    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704572    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704575    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704577    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704582    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704584    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704586    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704590    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704593    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704595    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704598    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704600    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704603    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704605    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704608    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704610    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704613    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704616    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704618    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:59:00.963778 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704621    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704624    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704626    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704631    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704634    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704637    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704640    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704642    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704645    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704648    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704664    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704668    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704671    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704674    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704677    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704680    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704682    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704686    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704689    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:59:00.964307 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704691    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704695    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704697    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704700    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704702    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704705    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704708    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704711    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704715    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704717    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704720    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704723    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704726    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704728    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704731    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704734    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704736    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704739    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:59:00.964912 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.704741    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.704746    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.712028    2579 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.712041    2579 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.712059    2579 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.712067    2579 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712122    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712127    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712130    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712133    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712136    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712139    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712142    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712145    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712148    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:59:00.965409 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712150    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712153    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712160    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712165    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712168    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712171    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712174    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712177    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712181    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712184    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712187    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712190    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712193    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712195    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712198    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712200    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712203    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712205    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712208    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:59:00.965969 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712211    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712213    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712216    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712218    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712221    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712223    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712226    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712228    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712231    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712234    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712237    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712241    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712243    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712246    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712248    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712253    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712258    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712260    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712263    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:59:00.966493 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712265    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712268    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712270    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712273    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712275    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712278    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712281    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712283    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712286    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712289    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712291    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712294    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712297    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712299    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712302    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712304    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712307    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712309    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712312    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712315    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:59:00.967120 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712317    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712319    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712322    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712324    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712327    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712333    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712336    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712338    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712342    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712346    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712348    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712351    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712354    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712356    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712359    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712361    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712364    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712367    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712369    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712372    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:59:00.967679 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712375    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712377    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712380    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712382    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712385    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712387    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712390    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712392    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712395    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712397    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712400    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712402    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.712407    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712532    2579 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712536    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 18:59:00.968265 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712540    2579 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712543    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712545    2579 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712548    2579 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712551    2579 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712556    2579 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712558    2579 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712563    2579 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712565    2579 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712568    2579 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712570    2579 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712573    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712575    2579 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712578    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712580    2579 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712583    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712585    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712588    2579 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712591    2579 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712593    2579 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 18:59:00.968731 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712596    2579 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712599    2579 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712603    2579 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712607    2579 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712610    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712614    2579 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712618    2579 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712622    2579 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712626    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712631    2579 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712634    2579 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712636    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712639    2579 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712641    2579 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712643    2579 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712646    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712649    2579 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712664    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712667    2579 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 18:59:00.969314 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712672    2579 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712676    2579 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712680    2579 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712685    2579 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712689    2579 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712693    2579 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712696    2579 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712701    2579 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712705    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712708    2579 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712712    2579 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712717    2579 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712720    2579 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712723    2579 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712725    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712728    2579 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712730    2579 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712733    2579 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712735    2579 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 18:59:00.969842 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712739    2579 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712742    2579 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712745    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712748    2579 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712751    2579 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712753    2579 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712756    2579 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712758    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712761    2579 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712764    2579 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712766    2579 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712771    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712773    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712777    2579 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712780    2579 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712782    2579 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712785    2579 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712787    2579 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712790    2579 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712792    2579 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 18:59:00.970404 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712795    2579 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712797    2579 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712799    2579 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712802    2579 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712805    2579 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712808    2579 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712810    2579 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712813    2579 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712815    2579 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712818    2579 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712820    2579 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712823    2579 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712825    2579 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712828    2579 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712830    2579 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712833    2579 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712835    2579 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712839    2579 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 18:59:00.970979 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:00.712842    2579 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.712846    2579 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.712944    2579 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.714819    2579 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.715993    2579 server.go:1019] "Starting client certificate rotation"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.716091    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.716127    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.718523    2579 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.725077    2579 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.733768    2579 log.go:25] "Validated CRI v1 runtime API"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.740803    2579 log.go:25] "Validated CRI v1 image API"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.743569    2579 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 18:59:00.971500 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.749114    2579 fs.go:135] Filesystem UUIDs: map[0eb07d56-8ae3-45b5-bf5f-ccac22df64af:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 b521ea89-b34e-4ed9-a978-6744e773ecb0:/dev/nvme0n1p3]
Jul 06 18:59:00.971911 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.749131    2579 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 18:59:00.971911 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.749494    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.755855    2579 manager.go:216] Machine: {Timestamp:2026-07-06 18:59:00.754540062 +0000 UTC m=+0.365865591 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3108206 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec25c076d69c6c7d6b270bd6a01db996 SystemUUID:ec25c076-d69c-6c7d-6b27-0bd6a01db996 BootID:384c7e3e-8afd-499f-bb68-08f2b9f16b95 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:eb:69:ca:6f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:eb:69:ca:6f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:d5:01:62:6a:7b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.755949    2579 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.756014    2579 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.758432    2579 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.758451    2579 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-165.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.760067    2579 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.760077    2579 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.760084    2579 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.761824    2579 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.763179    2579 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.766757    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.766867    2579 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.766892    2579 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.766908    2579 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.766921    2579 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.766933    2579 kubelet.go:398] "Adding apiserver pod source"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.766941    2579 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 18:59:01.094511 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.767202    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.767222    2579 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.769685    2579 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.771691    2579 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.771726    2579 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772805    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772823    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772831    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772836    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772845    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772850    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772858    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772863    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772870    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772875    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772883    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.772888    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.774107    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.774115    2579 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.777284    2579 server.go:1268] "Started kubelet"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.778450    2579 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.778740    2579 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.778776    2579 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.780171    2579 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-165.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.780497    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.780497    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.781855    2579 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.782242    2579 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.788125    2579 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.790170    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-165.ec2.internal.18bfc86b552084fb  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-165.ec2.internal,UID:ip-10-0-142-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-165.ec2.internal,},FirstTimestamp:2026-07-06 18:59:00.777256187 +0000 UTC m=+0.388581719,LastTimestamp:2026-07-06 18:59:00.777256187 +0000 UTC m=+0.388581719,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-165.ec2.internal,}"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.791922    2579 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.791934    2579 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.792055    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792619    2579 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792626    2579 factory.go:55] Registering systemd factory
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792642    2579 factory.go:223] Registration of the systemd container factory successfully
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792651    2579 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792632    2579 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792758    2579 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792765    2579 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792865    2579 factory.go:153] Registering CRI-O factory
Jul 06 18:59:01.095322 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792879    2579 factory.go:223] Registration of the crio container factory successfully
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792945    2579 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792962    2579 factory.go:103] Registering Raw factory
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.792972    2579 manager.go:1196] Started watching for new ooms in manager
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.793492    2579 manager.go:349] Starting recovery of all containers
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.803852    2579 manager.go:354] Recovery completed
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.806092    2579 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-165.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.806246    2579 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.809096    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.814270    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.814306    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.814315    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.814798    2579 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.814808    2579 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.814813    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.814822    2579 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.815488    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.815524    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.815536    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.816232    2579 policy_none.go:49] "None policy: Start"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.816243    2579 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.816252    2579 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.817453    2579 policy_none.go:47] "Start"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.820670    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-165.ec2.internal.18bfc86b5755abef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-165.ec2.internal,UID:ip-10-0-142-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-165.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-165.ec2.internal,},FirstTimestamp:2026-07-06 18:59:00.814293999 +0000 UTC m=+0.425619527,LastTimestamp:2026-07-06 18:59:00.814293999 +0000 UTC m=+0.425619527,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-165.ec2.internal,}"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.834383    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-165.ec2.internal.18bfc86b5755eddf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-165.ec2.internal,UID:ip-10-0-142-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-165.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-165.ec2.internal,},FirstTimestamp:2026-07-06 18:59:00.814310879 +0000 UTC m=+0.425636408,LastTimestamp:2026-07-06 18:59:00.814310879 +0000 UTC m=+0.425636408,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-165.ec2.internal,}"
Jul 06 18:59:01.099079 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.845650    2579 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-165.ec2.internal.18bfc86b57560f26  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-165.ec2.internal,UID:ip-10-0-142-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-142-165.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-142-165.ec2.internal,},FirstTimestamp:2026-07-06 18:59:00.814319398 +0000 UTC m=+0.425644927,LastTimestamp:2026-07-06 18:59:00.814319398 +0000 UTC m=+0.425644927,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-165.ec2.internal,}"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.852151    2579 manager.go:337] "Starting Device Plugin manager"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.852174    2579 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.852183    2579 server.go:87] "Starting device plugin registration server"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.852384    2579 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.852395    2579 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.852468    2579 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.852546    2579 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.852556    2579 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.853076    2579 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.853114    2579 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.856012    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wnfss"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.857769    2579 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-165.ec2.internal.18bfc86b5755abef\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-165.ec2.internal.18bfc86b5755abef  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-165.ec2.internal,UID:ip-10-0-142-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-165.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-165.ec2.internal,},FirstTimestamp:2026-07-06 18:59:00.814293999 +0000 UTC m=+0.425619527,LastTimestamp:2026-07-06 18:59:00.815511734 +0000 UTC m=+0.426837263,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-165.ec2.internal,}"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.871434    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wnfss"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.873856    2579 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-142-165.ec2.internal.18bfc86b5755eddf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-165.ec2.internal.18bfc86b5755eddf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-165.ec2.internal,UID:ip-10-0-142-165.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-165.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-165.ec2.internal,},FirstTimestamp:2026-07-06 18:59:00.814310879 +0000 UTC m=+0.425636408,LastTimestamp:2026-07-06 18:59:00.815530109 +0000 UTC m=+0.426855637,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-165.ec2.internal,}"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.920068    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.921230    2579 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.921262    2579 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.921279    2579 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.921308    2579 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.924102    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.953500    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.954290    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.954315    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.954326    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.954346    2579 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:00.966569    2579 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:00.966585    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-165.ec2.internal\": node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.004759    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.021749    2579 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"]
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.021796    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:01.100100 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.022529    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.022554    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.022570    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.023758    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.023927    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.023949    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.025213    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.025235    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.025248    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.025253    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.025271    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.025283    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.026497    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.026520    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.027232    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.027256    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.027265    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.065230    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-165.ec2.internal\" not found" node="ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.069326    2579 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-165.ec2.internal\" not found" node="ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.093990    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a9b387f1ea494ba8e3440d0784de7c5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-165.ec2.internal\" (UID: \"7a9b387f1ea494ba8e3440d0784de7c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.094011    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f74042b81dfcc4a8623a871af2883386-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"f74042b81dfcc4a8623a871af2883386\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.101129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.094044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f74042b81dfcc4a8623a871af2883386-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"f74042b81dfcc4a8623a871af2883386\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.105285    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.195044    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a9b387f1ea494ba8e3440d0784de7c5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-165.ec2.internal\" (UID: \"7a9b387f1ea494ba8e3440d0784de7c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.195069    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f74042b81dfcc4a8623a871af2883386-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"f74042b81dfcc4a8623a871af2883386\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.195089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f74042b81dfcc4a8623a871af2883386-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"f74042b81dfcc4a8623a871af2883386\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.195127    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f74042b81dfcc4a8623a871af2883386-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"f74042b81dfcc4a8623a871af2883386\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.195166    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7a9b387f1ea494ba8e3440d0784de7c5-config\") pod \"kube-apiserver-proxy-ip-10-0-142-165.ec2.internal\" (UID: \"7a9b387f1ea494ba8e3440d0784de7c5\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.195191    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f74042b81dfcc4a8623a871af2883386-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal\" (UID: \"f74042b81dfcc4a8623a871af2883386\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.284872 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.205749    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.476250 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.306826    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.476250 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.373017    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.476250 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.373018    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:01.476250 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.407789    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.672269 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.508938    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.672269 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.609473    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.832582 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.709584    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:01.832582 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.716182    2579 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 18:59:01.832582 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.716299    2579 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 18:59:01.832582 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.809625    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.878945    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 18:54:01 +0000 UTC" deadline="2028-03-19 22:12:27.38106288 +0000 UTC"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.878975    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14931h13m25.50209119s"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:01.910299    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:01.941016    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a9b387f1ea494ba8e3440d0784de7c5.slice/crio-b6a901eb478a14b8b284741b9cc68834d4756fcea2a7077aa82d1fda04a31091 WatchSource:0}: Error finding container b6a901eb478a14b8b284741b9cc68834d4756fcea2a7077aa82d1fda04a31091: Status 404 returned error can't find the container with id b6a901eb478a14b8b284741b9cc68834d4756fcea2a7077aa82d1fda04a31091
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:01.941296    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf74042b81dfcc4a8623a871af2883386.slice/crio-9b35dff6eaa9471aed287a8faafd15c7cf2d7285be6ec21011fa2b5133733c85 WatchSource:0}: Error finding container 9b35dff6eaa9471aed287a8faafd15c7cf2d7285be6ec21011fa2b5133733c85: Status 404 returned error can't find the container with id 9b35dff6eaa9471aed287a8faafd15c7cf2d7285be6ec21011fa2b5133733c85
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.947327    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.959720    2579 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:01.990524    2579 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.001787    2579 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.002982    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.003009    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.003020    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeHasSufficientPID"
Jul 06 18:59:02.032843 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:02.010336    2579 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-165.ec2.internal\" not found"
Jul 06 18:59:02.220298 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.051626    2579 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:59:02.220298 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.094188    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:02.220298 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.112510    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 18:59:02.220298 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.113554    2579 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal"
Jul 06 18:59:02.220298 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.125026    2579 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 18:59:02.757461 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.757437    2579 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 18:59:02.768077 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.768054    2579 apiserver.go:52] "Watching apiserver"
Jul 06 18:59:02.783818 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.783793    2579 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 18:59:02.784338 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.784308    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zmcn6","kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6","openshift-cluster-node-tuning-operator/tuned-hdz9f","openshift-dns/node-resolver-twkgb","openshift-image-registry/node-ca-cxvwk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal","openshift-multus/multus-additional-cni-plugins-6zmw5","kube-system/konnectivity-agent-jgr24","openshift-multus/multus-x25xs","openshift-multus/network-metrics-daemon-rspnw","openshift-network-diagnostics/network-check-target-x44nm","openshift-network-operator/iptables-alerter-b5pdt","openshift-ovn-kubernetes/ovnkube-node-zw8pk"]
Jul 06 18:59:02.786163 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.786138    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.788406 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.787871    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.789367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.789345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.790193 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.790122    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qxkdf\""
Jul 06 18:59:02.790559 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.790528    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.793431 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.792012    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 18:59:02.793431 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.792438    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 18:59:02.793431 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.793126    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 18:59:02.793633 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.793564    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 18:59:02.793633 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.793593    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 18:59:02.793891 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.793874    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 18:59:02.793960 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.793915    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 18:59:02.795201 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.794249    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-spxrn\""
Jul 06 18:59:02.795201 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.794377    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-kc5w5\""
Jul 06 18:59:02.795201 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.794627    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 18:59:02.795201 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.794722    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 18:59:02.795446 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.795283    2579 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 18:59:02.795446 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.795345    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.796665 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.796645    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.796825 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.796804    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 18:59:02.798217 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.797802    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:02.799462 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.799445    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 18:59:02.799953 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.799929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.802020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.801047    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 18:59:02.802020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.801125    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 18:59:02.802020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.801438    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.802020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.801482    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h4dbz\""
Jul 06 18:59:02.802020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.801780    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 18:59:02.802020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.801811    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 18:59:02.802020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.801879    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802055    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-lib-modules\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-registration-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802129    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-daemon-config\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802167    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-multus-certs\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802224    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlbtp\" (UniqueName: \"kubernetes.io/projected/98d66ddb-03be-4f67-a9d4-ae8720b77186-kube-api-access-rlbtp\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysctl-d\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-host\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802294    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-host-slash\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802318    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2hvb\" (UniqueName: \"kubernetes.io/projected/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-kube-api-access-c2hvb\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802273    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7kq2n\""
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802342    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8025aa-247e-4f49-b65e-0830d654dc2b-tmp-dir\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802379    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-system-cni-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.802398 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802321    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-os-release\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802438    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-socket-dir-parent\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802321    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wz5kg\""
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802463    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-var-lib-kubelet\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802485    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe49c3c1-b62f-4811-a096-53bd3d380508-tmp\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802508    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-device-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802534    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-cni-bin\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-cni-multus\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-hostroot\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802626    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-etc-kubernetes\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802642    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysconfig\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802662    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-systemd\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802682    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-tuned\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802723    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbbh5\" (UniqueName: \"kubernetes.io/projected/7f8025aa-247e-4f49-b65e-0830d654dc2b-kube-api-access-qbbh5\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-cni-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802779    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-kubelet\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.803084 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802805    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-conf-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802839    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-run\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802863    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-iptables-alerter-script\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802887    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802925    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-netns\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802958    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-modprobe-d\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.802987    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysctl-conf\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803012    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c7a182e-597b-43d5-9c20-9e10ee7422c6-host\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803038    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mm9z5\" (UniqueName: \"kubernetes.io/projected/7c7a182e-597b-43d5-9c20-9e10ee7422c6-kube-api-access-mm9z5\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803067    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803104    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zncvw\" (UniqueName: \"kubernetes.io/projected/90315c02-b9dd-4b51-ab65-3ad2df1829fb-kube-api-access-zncvw\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803132    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f8025aa-247e-4f49-b65e-0830d654dc2b-hosts-file\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803155    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98d66ddb-03be-4f67-a9d4-ae8720b77186-cni-binary-copy\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7c7a182e-597b-43d5-9c20-9e10ee7422c6-serviceca\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803219    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-sys-fs\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-sys\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.804054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803269    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kr2lg\" (UniqueName: \"kubernetes.io/projected/fe49c3c1-b62f-4811-a096-53bd3d380508-kube-api-access-kr2lg\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-socket-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803315    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d3ea8d3-5393-4900-8d57-3cc39e7fca96-agent-certs\") pod \"konnectivity-agent-jgr24\" (UID: \"2d3ea8d3-5393-4900-8d57-3cc39e7fca96\") " pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803338    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ea8d3-5393-4900-8d57-3cc39e7fca96-konnectivity-ca\") pod \"konnectivity-agent-jgr24\" (UID: \"2d3ea8d3-5393-4900-8d57-3cc39e7fca96\") " pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803346    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-cnibin\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803387    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-k8s-cni-cncf-io\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.803413    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-kubernetes\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:02.803439    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:02.804704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.804572    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 18:59:02.805171 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.805035    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:02.805171 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:02.805098    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:02.805453 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.805431    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 18:59:02.807468 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.806898    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:02.807468 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:02.806966    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:02.807952 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.807929    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-9wfqv\""
Jul 06 18:59:02.808042 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.807986    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 18:59:02.808109 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.808047    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 18:59:02.808757 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.808262    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 18:59:02.809863 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.809363    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 18:59:02.809863 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.809406    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 18:59:02.809863 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.809421    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 18:59:02.809863 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.809585    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wxkcb\""
Jul 06 18:59:02.809863 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.809747    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 18:59:02.809863 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.809841    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 18:59:02.810220 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.809844    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-czhzt\""
Jul 06 18:59:02.820440 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.820417    2579 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 18:59:02.867909 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.867891    2579 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rskm7"
Jul 06 18:59:02.876659 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.876640    2579 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rskm7"
Jul 06 18:59:02.893982 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.893960    2579 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 18:59:02.903827 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903808    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-host-slash\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.903927 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2hvb\" (UniqueName: \"kubernetes.io/projected/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-kube-api-access-c2hvb\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.903927 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903871    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8025aa-247e-4f49-b65e-0830d654dc2b-tmp-dir\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.903927 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903897    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-system-cni-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.903927 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903907    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-host-slash\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.903927 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-socket-dir-parent\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903949    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-cni-bin\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.903972    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-etc-kubernetes\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904000    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-systemd\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904027    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe49c3c1-b62f-4811-a096-53bd3d380508-tmp\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904053    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-var-lib-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904078    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovnkube-script-lib\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904101    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03aed337-be99-4849-94f5-0cdeeab4857f-dbus\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysconfig\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-cni-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904174    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-run-ovn-kubernetes\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904210    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-iptables-alerter-script\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.904299 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904275    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-etc-kubernetes\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904323    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-cni-bin\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904376    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-cni-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904425    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-system-cni-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904470    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-socket-dir-parent\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904544    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysconfig\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904572    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-netns\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904605    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904681    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-netns\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904695    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-modprobe-d\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904736    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c7a182e-597b-43d5-9c20-9e10ee7422c6-host\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mm9z5\" (UniqueName: \"kubernetes.io/projected/7c7a182e-597b-43d5-9c20-9e10ee7422c6-kube-api-access-mm9z5\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904783    2579 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904793    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c7a182e-597b-43d5-9c20-9e10ee7422c6-host\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904801    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7f8025aa-247e-4f49-b65e-0830d654dc2b-tmp-dir\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904838    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904792    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904879    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98d66ddb-03be-4f67-a9d4-ae8720b77186-cni-binary-copy\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.905131 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904909    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-system-cni-dir\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904903    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-iptables-alerter-script\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904947    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnn9h\" (UniqueName: \"kubernetes.io/projected/b989ffe7-f976-47d0-a1b9-de921bd55078-kube-api-access-cnn9h\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904805    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-modprobe-d\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904974    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7c7a182e-597b-43d5-9c20-9e10ee7422c6-serviceca\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.904999    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-slash\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905022    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-run-netns\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905046    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-etc-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905068    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-cni-bin\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905094    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-os-release\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-sys\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905174    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-socket-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905198    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-cnibin\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905202    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-sys\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-kubernetes\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905248    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-registration-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.906054 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905261    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-cnibin\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905274    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-multus-certs\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905299    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlbtp\" (UniqueName: \"kubernetes.io/projected/98d66ddb-03be-4f67-a9d4-ae8720b77186-kube-api-access-rlbtp\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905302    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-kubernetes\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905324    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905350    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-multus-certs\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905298    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-socket-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905361    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7c7a182e-597b-43d5-9c20-9e10ee7422c6-serviceca\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905350    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-node-log\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905353    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-registration-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905407    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-log-socket\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905439    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/98d66ddb-03be-4f67-a9d4-ae8720b77186-cni-binary-copy\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905486    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysctl-d\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-host\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905549    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-os-release\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-cni-multus\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905589    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-host\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovnkube-config\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.906931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905628    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-env-overrides\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905633    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-cni-multus\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905645    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysctl-d\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905654    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905690    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-os-release\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905701    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-var-lib-kubelet\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905727    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-device-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905750    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-hostroot\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905775    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-systemd\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905807    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-hostroot\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905815    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-var-lib-kubelet\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-tuned\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905864    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbbh5\" (UniqueName: \"kubernetes.io/projected/7f8025aa-247e-4f49-b65e-0830d654dc2b-kube-api-access-qbbh5\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905863    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-device-dir\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905878    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-systemd\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905891    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-kubelet\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.908059 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905914    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-conf-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905935    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905962    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-run\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.905986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906009    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-systemd-units\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906035    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03aed337-be99-4849-94f5-0cdeeab4857f-kubelet-config\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906059    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysctl-conf\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zncvw\" (UniqueName: \"kubernetes.io/projected/90315c02-b9dd-4b51-ab65-3ad2df1829fb-kube-api-access-zncvw\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906110    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f8025aa-247e-4f49-b65e-0830d654dc2b-hosts-file\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906123    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-run\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906137    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906162    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8r7q\" (UniqueName: \"kubernetes.io/projected/9209cc50-50be-47ab-b607-82201bf470a0-kube-api-access-q8r7q\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906174    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-var-lib-kubelet\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-sys-fs\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906224    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-conf-dir\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906246    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-sys-fs\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906254    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovn-node-metrics-cert\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.909329 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906291    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbw8k\" (UniqueName: \"kubernetes.io/projected/84ffbb4b-a820-4485-94c5-22cf32834cfe-kube-api-access-jbw8k\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906321    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kr2lg\" (UniqueName: \"kubernetes.io/projected/fe49c3c1-b62f-4811-a096-53bd3d380508-kube-api-access-kr2lg\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d3ea8d3-5393-4900-8d57-3cc39e7fca96-agent-certs\") pod \"konnectivity-agent-jgr24\" (UID: \"2d3ea8d3-5393-4900-8d57-3cc39e7fca96\") " pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906375    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ea8d3-5393-4900-8d57-3cc39e7fca96-konnectivity-ca\") pod \"konnectivity-agent-jgr24\" (UID: \"2d3ea8d3-5393-4900-8d57-3cc39e7fca96\") " pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-k8s-cni-cncf-io\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-daemon-config\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906453    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-kubelet\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906478    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-ovn\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906483    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/90315c02-b9dd-4b51-ab65-3ad2df1829fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906505    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-lib-modules\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906529    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-cni-netd\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906555    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-cnibin\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906581    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906608    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906712    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/98d66ddb-03be-4f67-a9d4-ae8720b77186-host-run-k8s-cni-cncf-io\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.906402    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-sysctl-conf\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.907085    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2d3ea8d3-5393-4900-8d57-3cc39e7fca96-konnectivity-ca\") pod \"konnectivity-agent-jgr24\" (UID: \"2d3ea8d3-5393-4900-8d57-3cc39e7fca96\") " pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:02.910314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.907223    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/98d66ddb-03be-4f67-a9d4-ae8720b77186-multus-daemon-config\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.911283 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.907371    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fe49c3c1-b62f-4811-a096-53bd3d380508-lib-modules\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.911283 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.907388    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7f8025aa-247e-4f49-b65e-0830d654dc2b-hosts-file\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.911283 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.908668    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fe49c3c1-b62f-4811-a096-53bd3d380508-tmp\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.911283 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.909638    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fe49c3c1-b62f-4811-a096-53bd3d380508-etc-tuned\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.911283 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.910390    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2d3ea8d3-5393-4900-8d57-3cc39e7fca96-agent-certs\") pod \"konnectivity-agent-jgr24\" (UID: \"2d3ea8d3-5393-4900-8d57-3cc39e7fca96\") " pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:02.916408 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.916348    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2hvb\" (UniqueName: \"kubernetes.io/projected/ecf857ea-7c7b-4956-80df-b3e7be3cfe1d-kube-api-access-c2hvb\") pod \"iptables-alerter-b5pdt\" (UID: \"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d\") " pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:02.922723 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.921295    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kr2lg\" (UniqueName: \"kubernetes.io/projected/fe49c3c1-b62f-4811-a096-53bd3d380508-kube-api-access-kr2lg\") pod \"tuned-hdz9f\" (UID: \"fe49c3c1-b62f-4811-a096-53bd3d380508\") " pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:02.922723 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.921936    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbbh5\" (UniqueName: \"kubernetes.io/projected/7f8025aa-247e-4f49-b65e-0830d654dc2b-kube-api-access-qbbh5\") pod \"node-resolver-twkgb\" (UID: \"7f8025aa-247e-4f49-b65e-0830d654dc2b\") " pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:02.922723 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.922604    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zncvw\" (UniqueName: \"kubernetes.io/projected/90315c02-b9dd-4b51-ab65-3ad2df1829fb-kube-api-access-zncvw\") pod \"aws-ebs-csi-driver-node-hfmb6\" (UID: \"90315c02-b9dd-4b51-ab65-3ad2df1829fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:02.923886 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.923850    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlbtp\" (UniqueName: \"kubernetes.io/projected/98d66ddb-03be-4f67-a9d4-ae8720b77186-kube-api-access-rlbtp\") pod \"multus-x25xs\" (UID: \"98d66ddb-03be-4f67-a9d4-ae8720b77186\") " pod="openshift-multus/multus-x25xs"
Jul 06 18:59:02.925013 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.924971    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mm9z5\" (UniqueName: \"kubernetes.io/projected/7c7a182e-597b-43d5-9c20-9e10ee7422c6-kube-api-access-mm9z5\") pod \"node-ca-cxvwk\" (UID: \"7c7a182e-597b-43d5-9c20-9e10ee7422c6\") " pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:02.926694 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.926647    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" event={"ID":"f74042b81dfcc4a8623a871af2883386","Type":"ContainerStarted","Data":"9b35dff6eaa9471aed287a8faafd15c7cf2d7285be6ec21011fa2b5133733c85"}
Jul 06 18:59:02.927994 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:02.927969    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal" event={"ID":"7a9b387f1ea494ba8e3440d0784de7c5","Type":"ContainerStarted","Data":"b6a901eb478a14b8b284741b9cc68834d4756fcea2a7077aa82d1fda04a31091"}
Jul 06 18:59:03.006932 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.006850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.006932 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.006896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-systemd-units\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.006932 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.006921    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03aed337-be99-4849-94f5-0cdeeab4857f-kubelet-config\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:03.007186 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.006948    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.007186 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007027    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-systemd-units\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007186 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007063    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/03aed337-be99-4849-94f5-0cdeeab4857f-kubelet-config\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:03.007186 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007033    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8r7q\" (UniqueName: \"kubernetes.io/projected/9209cc50-50be-47ab-b607-82201bf470a0-kube-api-access-q8r7q\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:03.007186 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007113    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovn-node-metrics-cert\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007186 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007164    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbw8k\" (UniqueName: \"kubernetes.io/projected/84ffbb4b-a820-4485-94c5-22cf32834cfe-kube-api-access-jbw8k\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-kubelet\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007291    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-ovn\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007316    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-cni-netd\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007345    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-cnibin\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007370    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007406    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007422    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-kubelet\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007438    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-systemd\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007480    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-var-lib-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.007505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovnkube-script-lib\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007535    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007570    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007697    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-tuning-conf-dir\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007765    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03aed337-be99-4849-94f5-0cdeeab4857f-dbus\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007772    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-ovn\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007581    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/03aed337-be99-4849-94f5-0cdeeab4857f-dbus\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007804    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-cni-netd\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-run-ovn-kubernetes\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007856    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-cnibin\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007862    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007872    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-var-lib-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007896    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-system-cni-dir\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007924    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnn9h\" (UniqueName: \"kubernetes.io/projected/b989ffe7-f976-47d0-a1b9-de921bd55078-kube-api-access-cnn9h\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007933    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-systemd\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007937    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-run-ovn-kubernetes\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007953    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-slash\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008007 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007982    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-system-cni-dir\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007986    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-run-netns\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.007993    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-slash\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008018    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-run-netns\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008020    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-etc-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-cni-bin\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.008091    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.008171    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs podName:9209cc50-50be-47ab-b607-82201bf470a0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:03.508148234 +0000 UTC m=+3.119473788 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs") pod "network-metrics-daemon-rspnw" (UID: "9209cc50-50be-47ab-b607-82201bf470a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008188    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-os-release\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008223    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008232    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008238    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-etc-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008247    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-node-log\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-log-socket\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008320    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovnkube-config\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008343    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b989ffe7-f976-47d0-a1b9-de921bd55078-os-release\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.008857 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008347    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-env-overrides\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008382    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008416    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008479    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovnkube-script-lib\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.008559    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.008611    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret podName:03aed337-be99-4849-94f5-0cdeeab4857f nodeName:}" failed. No retries permitted until 2026-07-06 18:59:03.508594132 +0000 UTC m=+3.119919654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret") pod "global-pull-secret-syncer-zmcn6" (UID: "03aed337-be99-4849-94f5-0cdeeab4857f") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008286    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-node-log\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008679    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-log-socket\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008288    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-host-cni-bin\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008745    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/84ffbb4b-a820-4485-94c5-22cf32834cfe-run-openvswitch\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008756    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-env-overrides\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.008969    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b989ffe7-f976-47d0-a1b9-de921bd55078-cni-binary-copy\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.009619 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.009182    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovnkube-config\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.011127 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.011103    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/84ffbb4b-a820-4485-94c5-22cf32834cfe-ovn-node-metrics-cert\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.021258 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.021236    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:59:03.021258 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.021261    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:59:03.021429 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.021274    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mkcfj for pod openshift-network-diagnostics/network-check-target-x44nm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:03.021429 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.021327    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj podName:9e0bcb48-8ed9-4de2-a59b-5db281e0059e nodeName:}" failed. No retries permitted until 2026-07-06 18:59:03.521313064 +0000 UTC m=+3.132638600 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mkcfj" (UniqueName: "kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj") pod "network-check-target-x44nm" (UID: "9e0bcb48-8ed9-4de2-a59b-5db281e0059e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:03.021545 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.021521    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8r7q\" (UniqueName: \"kubernetes.io/projected/9209cc50-50be-47ab-b607-82201bf470a0-kube-api-access-q8r7q\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:03.028844 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.028822    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnn9h\" (UniqueName: \"kubernetes.io/projected/b989ffe7-f976-47d0-a1b9-de921bd55078-kube-api-access-cnn9h\") pod \"multus-additional-cni-plugins-6zmw5\" (UID: \"b989ffe7-f976-47d0-a1b9-de921bd55078\") " pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.028995 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.028975    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbw8k\" (UniqueName: \"kubernetes.io/projected/84ffbb4b-a820-4485-94c5-22cf32834cfe-kube-api-access-jbw8k\") pod \"ovnkube-node-zw8pk\" (UID: \"84ffbb4b-a820-4485-94c5-22cf32834cfe\") " pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.098921 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.098897    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-x25xs"
Jul 06 18:59:03.111399 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.111380    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-b5pdt"
Jul 06 18:59:03.124188 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.124171    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6"
Jul 06 18:59:03.130080 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.130061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hdz9f"
Jul 06 18:59:03.137846 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.137821    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-twkgb"
Jul 06 18:59:03.147577 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.147556    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cxvwk"
Jul 06 18:59:03.154523 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.154502    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:03.161448 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.161425    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:03.168025 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.168009    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-6zmw5"
Jul 06 18:59:03.512797 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.512766    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:03.513004 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.512819    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:03.513004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.512942    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:03.513004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.513001    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret podName:03aed337-be99-4849-94f5-0cdeeab4857f nodeName:}" failed. No retries permitted until 2026-07-06 18:59:04.512986777 +0000 UTC m=+4.124312295 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret") pod "global-pull-secret-syncer-zmcn6" (UID: "03aed337-be99-4849-94f5-0cdeeab4857f") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:03.513153 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.512943    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:03.513153 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.513114    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs podName:9209cc50-50be-47ab-b607-82201bf470a0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:04.513092276 +0000 UTC m=+4.124417815 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs") pod "network-metrics-daemon-rspnw" (UID: "9209cc50-50be-47ab-b607-82201bf470a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:03.613502 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.613460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:03.613657 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.613606    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:59:03.613657 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.613622    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:59:03.613657 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.613632    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mkcfj for pod openshift-network-diagnostics/network-check-target-x44nm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:03.613799 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.613681    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj podName:9e0bcb48-8ed9-4de2-a59b-5db281e0059e nodeName:}" failed. No retries permitted until 2026-07-06 18:59:04.613668393 +0000 UTC m=+4.224993924 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mkcfj" (UniqueName: "kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj") pod "network-check-target-x44nm" (UID: "9e0bcb48-8ed9-4de2-a59b-5db281e0059e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:03.879827 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.879725    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:54:03 +0000 UTC" deadline="2028-01-16 13:43:15.101887785 +0000 UTC"
Jul 06 18:59:03.879827 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.879779    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13410h44m11.222112353s"
Jul 06 18:59:03.922531 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:03.922507    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:03.922675 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:03.922642    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:04.022623 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.022591    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f8025aa_247e_4f49_b65e_0830d654dc2b.slice/crio-5dd99e960362ecc957511feb3bd893f7a64ee6bc35a53f388081bcf525b7b59a WatchSource:0}: Error finding container 5dd99e960362ecc957511feb3bd893f7a64ee6bc35a53f388081bcf525b7b59a: Status 404 returned error can't find the container with id 5dd99e960362ecc957511feb3bd893f7a64ee6bc35a53f388081bcf525b7b59a
Jul 06 18:59:04.024770 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.024689    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb989ffe7_f976_47d0_a1b9_de921bd55078.slice/crio-6e5eaa80adacfe51aede74e2f380a1e7a2858834e3a27ff64795acf62000fad1 WatchSource:0}: Error finding container 6e5eaa80adacfe51aede74e2f380a1e7a2858834e3a27ff64795acf62000fad1: Status 404 returned error can't find the container with id 6e5eaa80adacfe51aede74e2f380a1e7a2858834e3a27ff64795acf62000fad1
Jul 06 18:59:04.028700 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.028674    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecf857ea_7c7b_4956_80df_b3e7be3cfe1d.slice/crio-f0baaabdec122a3a96c82929412e1107d9f14a1b53ce661eb729c3044e95fdaf WatchSource:0}: Error finding container f0baaabdec122a3a96c82929412e1107d9f14a1b53ce661eb729c3044e95fdaf: Status 404 returned error can't find the container with id f0baaabdec122a3a96c82929412e1107d9f14a1b53ce661eb729c3044e95fdaf
Jul 06 18:59:04.029597 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.029533    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98d66ddb_03be_4f67_a9d4_ae8720b77186.slice/crio-2f6899623107e3d2e225b1952a08ed11bc18882171078fbb6910580b368e0c73 WatchSource:0}: Error finding container 2f6899623107e3d2e225b1952a08ed11bc18882171078fbb6910580b368e0c73: Status 404 returned error can't find the container with id 2f6899623107e3d2e225b1952a08ed11bc18882171078fbb6910580b368e0c73
Jul 06 18:59:04.030310 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.030289    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d3ea8d3_5393_4900_8d57_3cc39e7fca96.slice/crio-4167b2650e2837cff4a6a0c462987516d741a7c1a418e18ac7ca1c98fab3c02c WatchSource:0}: Error finding container 4167b2650e2837cff4a6a0c462987516d741a7c1a418e18ac7ca1c98fab3c02c: Status 404 returned error can't find the container with id 4167b2650e2837cff4a6a0c462987516d741a7c1a418e18ac7ca1c98fab3c02c
Jul 06 18:59:04.031809 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.031634    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90315c02_b9dd_4b51_ab65_3ad2df1829fb.slice/crio-590e242c6d01b6a00d68a280858e18afe0f2687fec41c5812b185b0b5c60a756 WatchSource:0}: Error finding container 590e242c6d01b6a00d68a280858e18afe0f2687fec41c5812b185b0b5c60a756: Status 404 returned error can't find the container with id 590e242c6d01b6a00d68a280858e18afe0f2687fec41c5812b185b0b5c60a756
Jul 06 18:59:04.032760 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.032723    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe49c3c1_b62f_4811_a096_53bd3d380508.slice/crio-02fb91b3627d4fa405a0a13a60abc9bd19c23ed35962b07e4fbfa9f1e8c49d5e WatchSource:0}: Error finding container 02fb91b3627d4fa405a0a13a60abc9bd19c23ed35962b07e4fbfa9f1e8c49d5e: Status 404 returned error can't find the container with id 02fb91b3627d4fa405a0a13a60abc9bd19c23ed35962b07e4fbfa9f1e8c49d5e
Jul 06 18:59:04.034102 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.033764    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ffbb4b_a820_4485_94c5_22cf32834cfe.slice/crio-4e748061a0f3f487a401b787a0dbe0c18e83a9cf15a35e7cf607f8a22086d009 WatchSource:0}: Error finding container 4e748061a0f3f487a401b787a0dbe0c18e83a9cf15a35e7cf607f8a22086d009: Status 404 returned error can't find the container with id 4e748061a0f3f487a401b787a0dbe0c18e83a9cf15a35e7cf607f8a22086d009
Jul 06 18:59:04.036075 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:04.036048    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c7a182e_597b_43d5_9c20_9e10ee7422c6.slice/crio-4a98116dd2677e7c218cab5e0f4ef3877357505226bfa758b5e0b1d4e2132b71 WatchSource:0}: Error finding container 4a98116dd2677e7c218cab5e0f4ef3877357505226bfa758b5e0b1d4e2132b71: Status 404 returned error can't find the container with id 4a98116dd2677e7c218cab5e0f4ef3877357505226bfa758b5e0b1d4e2132b71
Jul 06 18:59:04.520564 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.520243    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:04.520774 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.520573    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:04.520774 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.520380    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:04.520914 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.520789    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs podName:9209cc50-50be-47ab-b607-82201bf470a0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:06.520768495 +0000 UTC m=+6.132094029 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs") pod "network-metrics-daemon-rspnw" (UID: "9209cc50-50be-47ab-b607-82201bf470a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:04.521198 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.520709    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:04.521198 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.521167    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret podName:03aed337-be99-4849-94f5-0cdeeab4857f nodeName:}" failed. No retries permitted until 2026-07-06 18:59:06.521150484 +0000 UTC m=+6.132476020 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret") pod "global-pull-secret-syncer-zmcn6" (UID: "03aed337-be99-4849-94f5-0cdeeab4857f") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:04.620965 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.620923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:04.621147 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.621128    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:59:04.621213 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.621156    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:59:04.621213 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.621170    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mkcfj for pod openshift-network-diagnostics/network-check-target-x44nm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:04.621313 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.621234    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj podName:9e0bcb48-8ed9-4de2-a59b-5db281e0059e nodeName:}" failed. No retries permitted until 2026-07-06 18:59:06.621213047 +0000 UTC m=+6.232538585 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mkcfj" (UniqueName: "kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj") pod "network-check-target-x44nm" (UID: "9e0bcb48-8ed9-4de2-a59b-5db281e0059e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:04.880413 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.880305    2579 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 18:54:03 +0000 UTC" deadline="2028-03-30 18:36:28.383582284 +0000 UTC"
Jul 06 18:59:04.880413 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.880344    2579 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15191h37m23.503244143s"
Jul 06 18:59:04.925588 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.924594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:04.925588 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.924724    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:04.925588 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.925145    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:04.925588 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:04.925252    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:04.943367 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.943334    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal" event={"ID":"7a9b387f1ea494ba8e3440d0784de7c5","Type":"ContainerStarted","Data":"4695757f6ff5b3b8fcbc09657aa55c4c27723f00d6cbb3ea696803f1f427bbac"}
Jul 06 18:59:04.958937 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.958898    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgr24" event={"ID":"2d3ea8d3-5393-4900-8d57-3cc39e7fca96","Type":"ContainerStarted","Data":"4167b2650e2837cff4a6a0c462987516d741a7c1a418e18ac7ca1c98fab3c02c"}
Jul 06 18:59:04.969778 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.969749    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b5pdt" event={"ID":"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d","Type":"ContainerStarted","Data":"f0baaabdec122a3a96c82929412e1107d9f14a1b53ce661eb729c3044e95fdaf"}
Jul 06 18:59:04.986970 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:04.986940    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerStarted","Data":"6e5eaa80adacfe51aede74e2f380a1e7a2858834e3a27ff64795acf62000fad1"}
Jul 06 18:59:05.000813 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.000477    2579 generic.go:356] "Generic (PLEG): container finished" podID="f74042b81dfcc4a8623a871af2883386" containerID="27c61fbc318bfb8633b5c47e410e4884332af5c5c5bbfd1200943b81f5bac492" exitCode=0
Jul 06 18:59:05.000813 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.000561    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" event={"ID":"f74042b81dfcc4a8623a871af2883386","Type":"ContainerDied","Data":"27c61fbc318bfb8633b5c47e410e4884332af5c5c5bbfd1200943b81f5bac492"}
Jul 06 18:59:05.009789 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.009761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxvwk" event={"ID":"7c7a182e-597b-43d5-9c20-9e10ee7422c6","Type":"ContainerStarted","Data":"4a98116dd2677e7c218cab5e0f4ef3877357505226bfa758b5e0b1d4e2132b71"}
Jul 06 18:59:05.019230 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.018075    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal" podStartSLOduration=3.018056495 podStartE2EDuration="3.018056495s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:04.96263972 +0000 UTC m=+4.573965264" watchObservedRunningTime="2026-07-06 18:59:05.018056495 +0000 UTC m=+4.629382037"
Jul 06 18:59:05.031682 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.031579    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"4e748061a0f3f487a401b787a0dbe0c18e83a9cf15a35e7cf607f8a22086d009"}
Jul 06 18:59:05.054745 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.048477    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hdz9f" event={"ID":"fe49c3c1-b62f-4811-a096-53bd3d380508","Type":"ContainerStarted","Data":"02fb91b3627d4fa405a0a13a60abc9bd19c23ed35962b07e4fbfa9f1e8c49d5e"}
Jul 06 18:59:05.060467 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.060392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6" event={"ID":"90315c02-b9dd-4b51-ab65-3ad2df1829fb","Type":"ContainerStarted","Data":"590e242c6d01b6a00d68a280858e18afe0f2687fec41c5812b185b0b5c60a756"}
Jul 06 18:59:05.071394 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.071360    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x25xs" event={"ID":"98d66ddb-03be-4f67-a9d4-ae8720b77186","Type":"ContainerStarted","Data":"2f6899623107e3d2e225b1952a08ed11bc18882171078fbb6910580b368e0c73"}
Jul 06 18:59:05.084870 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.084844    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twkgb" event={"ID":"7f8025aa-247e-4f49-b65e-0830d654dc2b","Type":"ContainerStarted","Data":"5dd99e960362ecc957511feb3bd893f7a64ee6bc35a53f388081bcf525b7b59a"}
Jul 06 18:59:05.922019 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:05.921982    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:05.922522 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:05.922117    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:06.104899 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:06.104810    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" event={"ID":"f74042b81dfcc4a8623a871af2883386","Type":"ContainerStarted","Data":"569c32a76003b118aae2cdd14b83c4303fb5358771ae0f32b3ae889f7bf21c64"}
Jul 06 18:59:06.547095 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:06.547060    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:06.547283 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:06.547126    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:06.547283 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.547251    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:06.547424 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.547311    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret podName:03aed337-be99-4849-94f5-0cdeeab4857f nodeName:}" failed. No retries permitted until 2026-07-06 18:59:10.547292852 +0000 UTC m=+10.158618388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret") pod "global-pull-secret-syncer-zmcn6" (UID: "03aed337-be99-4849-94f5-0cdeeab4857f") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:06.547724 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.547600    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:06.547724 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.547666    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs podName:9209cc50-50be-47ab-b607-82201bf470a0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:10.547649359 +0000 UTC m=+10.158974891 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs") pod "network-metrics-daemon-rspnw" (UID: "9209cc50-50be-47ab-b607-82201bf470a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:06.648339 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:06.648293    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:06.649004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.648516    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:59:06.649004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.648541    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:59:06.649004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.648555    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mkcfj for pod openshift-network-diagnostics/network-check-target-x44nm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:06.649004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.648618    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj podName:9e0bcb48-8ed9-4de2-a59b-5db281e0059e nodeName:}" failed. No retries permitted until 2026-07-06 18:59:10.648598469 +0000 UTC m=+10.259924002 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mkcfj" (UniqueName: "kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj") pod "network-check-target-x44nm" (UID: "9e0bcb48-8ed9-4de2-a59b-5db281e0059e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:06.923443 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:06.922815    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:06.923443 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.922983    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:06.923443 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:06.923025    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:06.923443 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:06.923122    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:07.922121 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:07.922087    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:07.922377 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:07.922197    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:08.923461 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:08.922867    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:08.923461 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:08.923006    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:08.924349 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:08.924153    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:08.924349 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:08.924289    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:09.922528 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:09.922295    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:09.922528 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:09.922428    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:10.584694 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:10.584647    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:10.585129 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:10.584725    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:10.585129 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.584773    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:10.585129 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.584840    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs podName:9209cc50-50be-47ab-b607-82201bf470a0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:18.584818907 +0000 UTC m=+18.196144438 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs") pod "network-metrics-daemon-rspnw" (UID: "9209cc50-50be-47ab-b607-82201bf470a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:10.585129 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.584852    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:10.585129 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.584901    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret podName:03aed337-be99-4849-94f5-0cdeeab4857f nodeName:}" failed. No retries permitted until 2026-07-06 18:59:18.584885707 +0000 UTC m=+18.196211238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret") pod "global-pull-secret-syncer-zmcn6" (UID: "03aed337-be99-4849-94f5-0cdeeab4857f") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:10.685843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:10.685802    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:10.685995 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.685952    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:59:10.685995 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.685973    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:59:10.685995 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.685986    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mkcfj for pod openshift-network-diagnostics/network-check-target-x44nm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:10.686159 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.686036    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj podName:9e0bcb48-8ed9-4de2-a59b-5db281e0059e nodeName:}" failed. No retries permitted until 2026-07-06 18:59:18.686019099 +0000 UTC m=+18.297344629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mkcfj" (UniqueName: "kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj") pod "network-check-target-x44nm" (UID: "9e0bcb48-8ed9-4de2-a59b-5db281e0059e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:10.922847 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:10.922756    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:10.923001 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.922868    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:10.923001 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:10.922934    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:10.923108 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:10.923021    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:11.922808 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:11.922325    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:11.922808 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:11.922444    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:12.921984 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:12.921946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:12.922161 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:12.921946    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:12.922161 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:12.922090    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:12.922161 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:12.922152    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:13.922137 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:13.922097    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:13.922624 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:13.922227    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:14.922070 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:14.922041    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:14.922186 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:14.922160    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:14.922542 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:14.922224    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:14.922542 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:14.922346    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:15.922411 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:15.922382    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:15.922848 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:15.922498    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:16.922396 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:16.921986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:16.922396 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:16.922116    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:16.922931 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:16.921986    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:16.922931 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:16.922605    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:17.922205 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:17.922169    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:17.922402 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:17.922286    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:18.647045 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:18.647019    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:18.647355 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:18.647068    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:18.647355 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.647150    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:18.647355 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.647153    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:18.647355 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.647196    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs podName:9209cc50-50be-47ab-b607-82201bf470a0 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:34.647181062 +0000 UTC m=+34.258506578 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs") pod "network-metrics-daemon-rspnw" (UID: "9209cc50-50be-47ab-b607-82201bf470a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:18.647355 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.647208    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret podName:03aed337-be99-4849-94f5-0cdeeab4857f nodeName:}" failed. No retries permitted until 2026-07-06 18:59:34.647202322 +0000 UTC m=+34.258527838 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret") pod "global-pull-secret-syncer-zmcn6" (UID: "03aed337-be99-4849-94f5-0cdeeab4857f") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:18.747376 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:18.747306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:18.747525 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.747478    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:59:18.747525 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.747499    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:59:18.747525 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.747511    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mkcfj for pod openshift-network-diagnostics/network-check-target-x44nm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:18.747763 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.747573    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj podName:9e0bcb48-8ed9-4de2-a59b-5db281e0059e nodeName:}" failed. No retries permitted until 2026-07-06 18:59:34.747555418 +0000 UTC m=+34.358880950 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mkcfj" (UniqueName: "kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj") pod "network-check-target-x44nm" (UID: "9e0bcb48-8ed9-4de2-a59b-5db281e0059e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:18.922452 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:18.921942    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:18.922452 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.922076    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:18.922452 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:18.921939    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:18.922452 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:18.922386    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:19.922234 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:19.922200    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:19.922700 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:19.922306    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:20.923051 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:20.923020    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:20.923489 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:20.923123    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:20.923489 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:20.923212    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:20.923489 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:20.923334    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:21.922486 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:21.922450    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:21.922644 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:21.922580    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:22.921945 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:22.921756    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:22.922373 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:22.921860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:22.922373 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:22.922161    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:22.922373 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:22.922046    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:23.140178 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.140001    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6" event={"ID":"90315c02-b9dd-4b51-ab65-3ad2df1829fb","Type":"ContainerStarted","Data":"6fd5f17ca7ba80dddab4214bfd13b5b240c3e3117d3d3c3d1caf48d3b9767f99"}
Jul 06 18:59:23.141434 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.141409    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-x25xs" event={"ID":"98d66ddb-03be-4f67-a9d4-ae8720b77186","Type":"ContainerStarted","Data":"0657dd6fdd3ce25fee542c63d164a814fa466692a0d175734df97ce498ce1c71"}
Jul 06 18:59:23.142824 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.142803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-twkgb" event={"ID":"7f8025aa-247e-4f49-b65e-0830d654dc2b","Type":"ContainerStarted","Data":"b10c0d07713997ab59211be0244845325b2bb083b88d96e586946c379d82f46b"}
Jul 06 18:59:23.144119 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.144099    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-jgr24" event={"ID":"2d3ea8d3-5393-4900-8d57-3cc39e7fca96","Type":"ContainerStarted","Data":"cf4241bae672f42429fcb4f3a145ed4c45b6ff5213c5fb3d715d4e67437203ee"}
Jul 06 18:59:23.145543 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.145522    2579 generic.go:356] "Generic (PLEG): container finished" podID="b989ffe7-f976-47d0-a1b9-de921bd55078" containerID="6bbbf54ca28339413b70b6f64091dc460eb24a5c3949ade2a1d233887a05b5de" exitCode=0
Jul 06 18:59:23.145625 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.145595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerDied","Data":"6bbbf54ca28339413b70b6f64091dc460eb24a5c3949ade2a1d233887a05b5de"}
Jul 06 18:59:23.147073 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.147041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cxvwk" event={"ID":"7c7a182e-597b-43d5-9c20-9e10ee7422c6","Type":"ContainerStarted","Data":"51c4416ba4ceea650f19202fb82ad067ce1465430c304e6645fdabf48c8b1c5e"}
Jul 06 18:59:23.149241 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.149209    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"d132d9b428cd0ed4635d5d515fb304f21b15252968097f2156d385966bca7199"}
Jul 06 18:59:23.149330 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.149264    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"85b5ff1866848a58ed84c9afb6e5e83b987aa4b8f2334955d88a224eec5b61d1"}
Jul 06 18:59:23.149330 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.149296    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"cf047c91a2788b03c57e22cf5cf468dd34f92fe48947d02976a34ff9703bbb4d"}
Jul 06 18:59:23.150605 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.150585    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hdz9f" event={"ID":"fe49c3c1-b62f-4811-a096-53bd3d380508","Type":"ContainerStarted","Data":"39d3b31d64d53d6d07d454f64cba162a156a1a42d7539170bd79838cb248f9d2"}
Jul 06 18:59:23.162634 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.162595    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-x25xs" podStartSLOduration=3.367276349 podStartE2EDuration="22.162585135s" totalImagesPullingTime="18.795308786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:23.162398089 +0000 UTC m=+22.773723637" watchObservedRunningTime="2026-07-06 18:59:23.162585135 +0000 UTC m=+22.773910673"
Jul 06 18:59:23.163203 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.163172    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-165.ec2.internal" podStartSLOduration=21.163163775 podStartE2EDuration="21.163163775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:06.129906009 +0000 UTC m=+5.741231553" watchObservedRunningTime="2026-07-06 18:59:23.163163775 +0000 UTC m=+22.774489329"
Jul 06 18:59:23.211630 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.211592    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hdz9f" podStartSLOduration=3.466622219 podStartE2EDuration="22.211579338s" totalImagesPullingTime="18.744957119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:23.187885061 +0000 UTC m=+22.799210599" watchObservedRunningTime="2026-07-06 18:59:23.211579338 +0000 UTC m=+22.822904876"
Jul 06 18:59:23.230795 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.230764    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-twkgb" podStartSLOduration=3.480438009 podStartE2EDuration="22.23075152s" totalImagesPullingTime="18.750313511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:23.211733246 +0000 UTC m=+22.823058777" watchObservedRunningTime="2026-07-06 18:59:23.23075152 +0000 UTC m=+22.842077121"
Jul 06 18:59:23.251221 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.248837    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cxvwk" podStartSLOduration=3.511564164 podStartE2EDuration="22.248816849s" totalImagesPullingTime="18.737252685s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:23.230866044 +0000 UTC m=+22.842191565" watchObservedRunningTime="2026-07-06 18:59:23.248816849 +0000 UTC m=+22.860142384"
Jul 06 18:59:23.251221 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.249191    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-jgr24" podStartSLOduration=7.836791692 podStartE2EDuration="22.249178171s" totalImagesPullingTime="14.412386479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:23.248601294 +0000 UTC m=+22.859926833" watchObservedRunningTime="2026-07-06 18:59:23.249178171 +0000 UTC m=+22.860503712"
Jul 06 18:59:23.922046 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:23.922016    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:23.922892 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:23.922135    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:24.156178 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.156143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"cb42167d77d49bf9a67e12c50a679e8372f3c8774f0ddcfaff11f5a54f482ce6"}
Jul 06 18:59:24.156304 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.156225    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"47ce1957c6621fc994788c97773900c045c5ae7e6c46b13370cd39c50b5fe944"}
Jul 06 18:59:24.156304 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.156276    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"026010be23dedea238e19ea8fa4a9b73c9f7d5c6cb43a1902b8f6d1a44bc3678"}
Jul 06 18:59:24.157646 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.157599    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-b5pdt" event={"ID":"ecf857ea-7c7b-4956-80df-b3e7be3cfe1d","Type":"ContainerStarted","Data":"1365e71bc7b098a3b00d596b17653d36efa89204da2566c28747189ff903edb6"}
Jul 06 18:59:24.180866 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.180830    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-b5pdt" podStartSLOduration=4.436368898 podStartE2EDuration="23.180815297s" totalImagesPullingTime="18.744446399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:24.180429295 +0000 UTC m=+23.791754832" watchObservedRunningTime="2026-07-06 18:59:24.180815297 +0000 UTC m=+23.792140834"
Jul 06 18:59:24.222051 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.222020    2579 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 18:59:24.531089 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.531005    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:24.531549 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.531527    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:24.864994 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.864892    2579 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T18:59:24.222039724Z","UUID":"ccee723f-26d3-459a-9ede-3f4ccc995858","Handler":null,"Name":"","Endpoint":""}
Jul 06 18:59:24.866594 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.866564    2579 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 18:59:24.866733 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.866616    2579 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 18:59:24.921944 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.921921    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:24.922068 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:24.922031    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:24.922414 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:24.922372    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:24.922528 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:24.922501    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:25.161583 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:25.161511    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6" event={"ID":"90315c02-b9dd-4b51-ab65-3ad2df1829fb","Type":"ContainerStarted","Data":"47f40f283152d6e0695c64643b43ae4fd3598d81c450f745c76f3997cdb41146"}
Jul 06 18:59:25.161887 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:25.161867    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:25.162288 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:25.162274    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-jgr24"
Jul 06 18:59:25.922031 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:25.921833    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:25.922305 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:25.922114    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:26.167421 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:26.167386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"e00b1e545ef3db760ca2273e040fb75f2223abb21470455b149be7b3f2a8b96a"}
Jul 06 18:59:26.169704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:26.169674    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6" event={"ID":"90315c02-b9dd-4b51-ab65-3ad2df1829fb","Type":"ContainerStarted","Data":"deba2eb001a381304ce425f734dc70e202a2a12127427e999c5ac77d8c019322"}
Jul 06 18:59:26.192975 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:26.192892    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hfmb6" podStartSLOduration=4.380155903 podStartE2EDuration="25.192878656s" totalImagesPullingTime="20.812722753s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:26.191426196 +0000 UTC m=+25.802751735" watchObservedRunningTime="2026-07-06 18:59:26.192878656 +0000 UTC m=+25.804204173"
Jul 06 18:59:26.922413 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:26.922380    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:26.923004 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:26.922379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:26.923004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:26.922527    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:26.923004 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:26.922583    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:27.922292 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:27.922218    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:27.922453 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:27.922343    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:28.922300 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:28.922255    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:28.922445 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:28.922356    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:28.922445 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:28.922385    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:28.922445 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:28.922439    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:29.180143 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.180113    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" event={"ID":"84ffbb4b-a820-4485-94c5-22cf32834cfe","Type":"ContainerStarted","Data":"4f35fc9e0c452fa20b6f4df0e8c93630b395bbf7ce4f0a8d7164f257152526fb"}
Jul 06 18:59:29.180427 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.180295    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:29.180427 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.180373    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:29.181859 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.181833    2579 generic.go:356] "Generic (PLEG): container finished" podID="b989ffe7-f976-47d0-a1b9-de921bd55078" containerID="73c2f2459c44d0a18a1b6f4e07f15f2ef3151d7f7a4532900d5884bda5142ac5" exitCode=0
Jul 06 18:59:29.181977 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.181865    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerDied","Data":"73c2f2459c44d0a18a1b6f4e07f15f2ef3151d7f7a4532900d5884bda5142ac5"}
Jul 06 18:59:29.195834 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.195817    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:29.285456 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.285412    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk" podStartSLOduration=9.481643086 podStartE2EDuration="28.285400012s" totalImagesPullingTime="18.803756926s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:29.230034224 +0000 UTC m=+28.841359761" watchObservedRunningTime="2026-07-06 18:59:29.285400012 +0000 UTC m=+28.896725528"
Jul 06 18:59:29.921958 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:29.921920    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:29.922279 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:29.922039    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:30.184964 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.184884    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:30.200257 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.200231    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 18:59:30.835775 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.835593    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zmcn6"]
Jul 06 18:59:30.835941 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.835913    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:30.836054 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:30.836033    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:30.836439 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.836419    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x44nm"]
Jul 06 18:59:30.836513 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.836497    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:30.836584 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:30.836564    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:30.838446 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.838426    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rspnw"]
Jul 06 18:59:30.838522 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:30.838504    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:30.838582 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:30.838567    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:31.187861 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:31.187784    2579 generic.go:356] "Generic (PLEG): container finished" podID="b989ffe7-f976-47d0-a1b9-de921bd55078" containerID="f096b8be1899aee02fa08a82a856425def0402ed733c1ac9c1fa57d75ef7ba8c" exitCode=0
Jul 06 18:59:31.188227 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:31.187863    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerDied","Data":"f096b8be1899aee02fa08a82a856425def0402ed733c1ac9c1fa57d75ef7ba8c"}
Jul 06 18:59:31.921848 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:31.921813    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:31.921992 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:31.921917    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:32.921673 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:32.921599    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:32.922208 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:32.921600    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:32.922208 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:32.921711    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:32.922208 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:32.921759    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:33.194948 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:33.194878    2579 generic.go:356] "Generic (PLEG): container finished" podID="b989ffe7-f976-47d0-a1b9-de921bd55078" containerID="65bc1fc6b0e281c15d34e958168cd3735b14d6cfa746c7dc20c441f0be44cf39" exitCode=0
Jul 06 18:59:33.194948 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:33.194914    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerDied","Data":"65bc1fc6b0e281c15d34e958168cd3735b14d6cfa746c7dc20c441f0be44cf39"}
Jul 06 18:59:33.921671 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:33.921630    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:33.922085 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:33.921786    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:34.670010 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:34.669930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:34.670010 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:34.669992    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:34.670210 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.670085    2579 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:34.670210 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.670103    2579 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:34.670289 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.670224    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs podName:9209cc50-50be-47ab-b607-82201bf470a0 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:06.670207843 +0000 UTC m=+66.281533359 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs") pod "network-metrics-daemon-rspnw" (UID: "9209cc50-50be-47ab-b607-82201bf470a0") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 18:59:34.670289 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.670241    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret podName:03aed337-be99-4849-94f5-0cdeeab4857f nodeName:}" failed. No retries permitted until 2026-07-06 19:00:06.670234357 +0000 UTC m=+66.281559872 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret") pod "global-pull-secret-syncer-zmcn6" (UID: "03aed337-be99-4849-94f5-0cdeeab4857f") : object "kube-system"/"original-pull-secret" not registered
Jul 06 18:59:34.771185 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:34.771161    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:34.771369 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.771350    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 18:59:34.771428 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.771376    2579 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 18:59:34.771428 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.771390    2579 projected.go:196] Error preparing data for projected volume kube-api-access-mkcfj for pod openshift-network-diagnostics/network-check-target-x44nm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:34.771500 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.771446    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj podName:9e0bcb48-8ed9-4de2-a59b-5db281e0059e nodeName:}" failed. No retries permitted until 2026-07-06 19:00:06.771429465 +0000 UTC m=+66.382755000 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mkcfj" (UniqueName: "kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj") pod "network-check-target-x44nm" (UID: "9e0bcb48-8ed9-4de2-a59b-5db281e0059e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 18:59:34.921868 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:34.921794    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:34.922214 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.921910    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zmcn6" podUID="03aed337-be99-4849-94f5-0cdeeab4857f"
Jul 06 18:59:34.922214 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:34.921967    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:34.922214 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:34.922050    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x44nm" podUID="9e0bcb48-8ed9-4de2-a59b-5db281e0059e"
Jul 06 18:59:35.921779 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:35.921750    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:35.921951 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:35.921875    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rspnw" podUID="9209cc50-50be-47ab-b607-82201bf470a0"
Jul 06 18:59:36.661357 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.661291    2579 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-165.ec2.internal" event="NodeReady"
Jul 06 18:59:36.661494 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.661429    2579 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 18:59:36.730570 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.730533    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2nz9k"]
Jul 06 18:59:36.735667 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.735634    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:36.740052 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.740028    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74bc668d5c-lv5js"]
Jul 06 18:59:36.743616 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.743238    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.744585 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.744509    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 18:59:36.744704 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.744619    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r2qkq\""
Jul 06 18:59:36.744893 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.744878    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 18:59:36.745219 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.745194    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 18:59:36.745496 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.745476    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2ll9p"]
Jul 06 18:59:36.746565 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.746546    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 18:59:36.748061 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.748045    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 18:59:36.748314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.748291    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 18:59:36.748625 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.748606    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ch228\""
Jul 06 18:59:36.748904 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.748886    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.752421 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.752402    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 18:59:36.752503 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.752421    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 18:59:36.752799 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.752781    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2f7gj\""
Jul 06 18:59:36.753472 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.753451    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 18:59:36.758441 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.758422    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2nz9k"]
Jul 06 18:59:36.770399 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.770371    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ll9p"]
Jul 06 18:59:36.770485 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.770473    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74bc668d5c-lv5js"]
Jul 06 18:59:36.888234 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888198    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.888378 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888262    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-bound-sa-token\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.888378 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-image-registry-private-configuration\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.888378 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-installation-pull-secrets\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.888378 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888345    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-tmp-dir\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.888378 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkk7b\" (UniqueName: \"kubernetes.io/projected/d822dfd6-b5ec-4a8e-9fee-f5797dddac60-kube-api-access-qkk7b\") pod \"ingress-canary-2nz9k\" (UID: \"d822dfd6-b5ec-4a8e-9fee-f5797dddac60\") " pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:36.888627 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888391    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzbft\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-kube-api-access-dzbft\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.888627 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-config-volume\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.888627 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888450    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nmp9\" (UniqueName: \"kubernetes.io/projected/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-kube-api-access-8nmp9\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.888627 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15c3ec3f-f235-4123-8af1-227508a8821d-ca-trust-extracted\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.888627 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888582    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-trusted-ca\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.888627 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888616    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-registry-certificates\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.888871 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888673    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d822dfd6-b5ec-4a8e-9fee-f5797dddac60-cert\") pod \"ingress-canary-2nz9k\" (UID: \"d822dfd6-b5ec-4a8e-9fee-f5797dddac60\") " pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:36.888871 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.888699    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.922520 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.922444    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 18:59:36.922520 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.922463    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 18:59:36.925740 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.925716    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 18:59:36.926149 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.926131    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 18:59:36.926446 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.926425    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 18:59:36.926540 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.926442    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zn9xt\""
Jul 06 18:59:36.989212 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15c3ec3f-f235-4123-8af1-227508a8821d-ca-trust-extracted\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989338 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-trusted-ca\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989338 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989242    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-registry-certificates\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989338 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989258    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d822dfd6-b5ec-4a8e-9fee-f5797dddac60-cert\") pod \"ingress-canary-2nz9k\" (UID: \"d822dfd6-b5ec-4a8e-9fee-f5797dddac60\") " pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:36.989338 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989283    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989338 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989354    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-bound-sa-token\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989393    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-image-registry-private-configuration\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989418    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-installation-pull-secrets\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989439    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-tmp-dir\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkk7b\" (UniqueName: \"kubernetes.io/projected/d822dfd6-b5ec-4a8e-9fee-f5797dddac60-kube-api-access-qkk7b\") pod \"ingress-canary-2nz9k\" (UID: \"d822dfd6-b5ec-4a8e-9fee-f5797dddac60\") " pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989484    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzbft\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-kube-api-access-dzbft\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-config-volume\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989542    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15c3ec3f-f235-4123-8af1-227508a8821d-ca-trust-extracted\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.989598 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nmp9\" (UniqueName: \"kubernetes.io/projected/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-kube-api-access-8nmp9\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.990041 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.989898    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-registry-certificates\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.990136 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:36.990118    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:59:36.990193 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.990154    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-trusted-ca\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.990241 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.990187    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-tmp-dir\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.990241 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:36.990192    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls podName:eda345d4-11ac-4c1e-9e3b-1607b869c7c2 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:37.490173421 +0000 UTC m=+37.101498952 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls") pod "dns-default-2ll9p" (UID: "eda345d4-11ac-4c1e-9e3b-1607b869c7c2") : secret "dns-default-metrics-tls" not found
Jul 06 18:59:36.990241 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:36.990216    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:59:36.990241 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:36.990230    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bc668d5c-lv5js: secret "image-registry-tls" not found
Jul 06 18:59:36.990432 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:36.990280    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls podName:15c3ec3f-f235-4123-8af1-227508a8821d nodeName:}" failed. No retries permitted until 2026-07-06 18:59:37.490265543 +0000 UTC m=+37.101591059 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls") pod "image-registry-74bc668d5c-lv5js" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d") : secret "image-registry-tls" not found
Jul 06 18:59:36.990432 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.990394    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-config-volume\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:36.994010 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.993853    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-installation-pull-secrets\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.994090 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.993861    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-image-registry-private-configuration\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:36.994090 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:36.993931    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d822dfd6-b5ec-4a8e-9fee-f5797dddac60-cert\") pod \"ingress-canary-2nz9k\" (UID: \"d822dfd6-b5ec-4a8e-9fee-f5797dddac60\") " pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:37.000711 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.000688    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzbft\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-kube-api-access-dzbft\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:37.005115 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.005095    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-bound-sa-token\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:37.005731 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.005709    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nmp9\" (UniqueName: \"kubernetes.io/projected/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-kube-api-access-8nmp9\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:37.008212 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.008195    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkk7b\" (UniqueName: \"kubernetes.io/projected/d822dfd6-b5ec-4a8e-9fee-f5797dddac60-kube-api-access-qkk7b\") pod \"ingress-canary-2nz9k\" (UID: \"d822dfd6-b5ec-4a8e-9fee-f5797dddac60\") " pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:37.047723 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.047698    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2nz9k"
Jul 06 18:59:37.494575 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.494538    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:37.494764 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.494582    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:37.494764 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:37.494712    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:59:37.494764 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:37.494737    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bc668d5c-lv5js: secret "image-registry-tls" not found
Jul 06 18:59:37.494881 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:37.494760    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:59:37.494881 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:37.494804    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls podName:15c3ec3f-f235-4123-8af1-227508a8821d nodeName:}" failed. No retries permitted until 2026-07-06 18:59:38.494786399 +0000 UTC m=+38.106111915 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls") pod "image-registry-74bc668d5c-lv5js" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d") : secret "image-registry-tls" not found
Jul 06 18:59:37.494881 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:37.494818    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls podName:eda345d4-11ac-4c1e-9e3b-1607b869c7c2 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:38.49481217 +0000 UTC m=+38.106137685 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls") pod "dns-default-2ll9p" (UID: "eda345d4-11ac-4c1e-9e3b-1607b869c7c2") : secret "dns-default-metrics-tls" not found
Jul 06 18:59:37.592833 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.592807    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"]
Jul 06 18:59:37.596099 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.596076    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:37.601597 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.601573    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 06 18:59:37.602797 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.602774    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 06 18:59:37.602936 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.602780    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dll8m\""
Jul 06 18:59:37.607684 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.607647    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"]
Jul 06 18:59:37.697690 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.696354    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a54d672b-7076-4068-92ab-5e1a4406abc9-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:37.697690 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.696439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:37.780108 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.780076    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2nz9k"]
Jul 06 18:59:37.783867 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:37.783845    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd822dfd6_b5ec_4a8e_9fee_f5797dddac60.slice/crio-875e94938a17b1f53684b9f79a3608b9d8dfb8e73898ef51253e1940c8600767 WatchSource:0}: Error finding container 875e94938a17b1f53684b9f79a3608b9d8dfb8e73898ef51253e1940c8600767: Status 404 returned error can't find the container with id 875e94938a17b1f53684b9f79a3608b9d8dfb8e73898ef51253e1940c8600767
Jul 06 18:59:37.796785 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.796760    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:37.796913 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:37.796892    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 18:59:37.796913 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.796909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a54d672b-7076-4068-92ab-5e1a4406abc9-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:37.797039 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:37.796943    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert podName:a54d672b-7076-4068-92ab-5e1a4406abc9 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:38.296928469 +0000 UTC m=+37.908254007 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhr6r" (UID: "a54d672b-7076-4068-92ab-5e1a4406abc9") : secret "networking-console-plugin-cert" not found
Jul 06 18:59:37.797484 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.797462    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a54d672b-7076-4068-92ab-5e1a4406abc9-nginx-conf\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:37.922087 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.922066    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 18:59:37.925650 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.925627    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 18:59:37.925994 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:37.925702    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kw7z\""
Jul 06 18:59:38.207079 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:38.207048    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2nz9k" event={"ID":"d822dfd6-b5ec-4a8e-9fee-f5797dddac60","Type":"ContainerStarted","Data":"875e94938a17b1f53684b9f79a3608b9d8dfb8e73898ef51253e1940c8600767"}
Jul 06 18:59:38.209685 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:38.209644    2579 generic.go:356] "Generic (PLEG): container finished" podID="b989ffe7-f976-47d0-a1b9-de921bd55078" containerID="ff2897235bbfb65e6b18e0536d814080eb80687f85bc9f421dc59a1ae7db2272" exitCode=0
Jul 06 18:59:38.209789 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:38.209693    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerDied","Data":"ff2897235bbfb65e6b18e0536d814080eb80687f85bc9f421dc59a1ae7db2272"}
Jul 06 18:59:38.299941 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:38.299915    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:38.300066 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:38.300050    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 18:59:38.300126 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:38.300116    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert podName:a54d672b-7076-4068-92ab-5e1a4406abc9 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:39.300098357 +0000 UTC m=+38.911423879 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhr6r" (UID: "a54d672b-7076-4068-92ab-5e1a4406abc9") : secret "networking-console-plugin-cert" not found
Jul 06 18:59:38.501910 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:38.501686    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:38.501910 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:38.501876    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:38.502151 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:38.501840    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:59:38.502151 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:38.501948    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bc668d5c-lv5js: secret "image-registry-tls" not found
Jul 06 18:59:38.502151 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:38.502000    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls podName:15c3ec3f-f235-4123-8af1-227508a8821d nodeName:}" failed. No retries permitted until 2026-07-06 18:59:40.501985412 +0000 UTC m=+40.113310928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls") pod "image-registry-74bc668d5c-lv5js" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d") : secret "image-registry-tls" not found
Jul 06 18:59:38.502151 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:38.502034    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:59:38.502151 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:38.502083    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls podName:eda345d4-11ac-4c1e-9e3b-1607b869c7c2 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:40.50207061 +0000 UTC m=+40.113396142 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls") pod "dns-default-2ll9p" (UID: "eda345d4-11ac-4c1e-9e3b-1607b869c7c2") : secret "dns-default-metrics-tls" not found
Jul 06 18:59:39.215545 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:39.215503    2579 generic.go:356] "Generic (PLEG): container finished" podID="b989ffe7-f976-47d0-a1b9-de921bd55078" containerID="3ec3d952710c2f93ef63800fc284abd6b903805772de4c636ad02f63ee95ae57" exitCode=0
Jul 06 18:59:39.215972 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:39.215554    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerDied","Data":"3ec3d952710c2f93ef63800fc284abd6b903805772de4c636ad02f63ee95ae57"}
Jul 06 18:59:39.308903 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:39.308868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:39.309067 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:39.309015    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 18:59:39.309134 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:39.309085    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert podName:a54d672b-7076-4068-92ab-5e1a4406abc9 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:41.309064355 +0000 UTC m=+40.920389894 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhr6r" (UID: "a54d672b-7076-4068-92ab-5e1a4406abc9") : secret "networking-console-plugin-cert" not found
Jul 06 18:59:40.220074 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:40.220046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" event={"ID":"b989ffe7-f976-47d0-a1b9-de921bd55078","Type":"ContainerStarted","Data":"f13568b7cc336c0d00b29dfe2daa459d1ee7c0bcb3dc913e76aec09080b80f4a"}
Jul 06 18:59:40.517930 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:40.517854    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:40.517930 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:40.517886    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:40.518091 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:40.518007    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:59:40.518091 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:40.518032    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bc668d5c-lv5js: secret "image-registry-tls" not found
Jul 06 18:59:40.518091 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:40.518047    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:59:40.518177 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:40.518097    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls podName:15c3ec3f-f235-4123-8af1-227508a8821d nodeName:}" failed. No retries permitted until 2026-07-06 18:59:44.518077433 +0000 UTC m=+44.129402963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls") pod "image-registry-74bc668d5c-lv5js" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d") : secret "image-registry-tls" not found
Jul 06 18:59:40.518177 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:40.518111    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls podName:eda345d4-11ac-4c1e-9e3b-1607b869c7c2 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:44.518105091 +0000 UTC m=+44.129430606 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls") pod "dns-default-2ll9p" (UID: "eda345d4-11ac-4c1e-9e3b-1607b869c7c2") : secret "dns-default-metrics-tls" not found
Jul 06 18:59:41.222955 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:41.222923    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2nz9k" event={"ID":"d822dfd6-b5ec-4a8e-9fee-f5797dddac60","Type":"ContainerStarted","Data":"c200e26dfee06f29645f21ea5fcd3bcfc93a308f8400111392d6ccf5bf25bf73"}
Jul 06 18:59:41.249732 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:41.249685    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-6zmw5" podStartSLOduration=9.103387702 podStartE2EDuration="40.249652331s" totalImagesPullingTime="31.146264629s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:40.26775857 +0000 UTC m=+39.879084108" watchObservedRunningTime="2026-07-06 18:59:41.249652331 +0000 UTC m=+40.860977869"
Jul 06 18:59:41.250155 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:41.250128    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2nz9k" podStartSLOduration=2.8341819790000002 podStartE2EDuration="5.250120754s" totalImagesPullingTime="2.415938775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:41.249757961 +0000 UTC m=+40.861083500" watchObservedRunningTime="2026-07-06 18:59:41.250120754 +0000 UTC m=+40.861446293"
Jul 06 18:59:41.322733 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:41.322704    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:41.326085 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:41.323535    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 18:59:41.326085 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:41.324842    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert podName:a54d672b-7076-4068-92ab-5e1a4406abc9 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:45.324816565 +0000 UTC m=+44.936142087 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhr6r" (UID: "a54d672b-7076-4068-92ab-5e1a4406abc9") : secret "networking-console-plugin-cert" not found
Jul 06 18:59:44.541386 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:44.541350    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:44.541386 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:44.541384    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:44.541855 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:44.541504    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:59:44.541855 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:44.541519    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:59:44.541855 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:44.541528    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bc668d5c-lv5js: secret "image-registry-tls" not found
Jul 06 18:59:44.541855 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:44.541579    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls podName:eda345d4-11ac-4c1e-9e3b-1607b869c7c2 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:52.541563456 +0000 UTC m=+52.152888989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls") pod "dns-default-2ll9p" (UID: "eda345d4-11ac-4c1e-9e3b-1607b869c7c2") : secret "dns-default-metrics-tls" not found
Jul 06 18:59:44.541855 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:44.541593    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls podName:15c3ec3f-f235-4123-8af1-227508a8821d nodeName:}" failed. No retries permitted until 2026-07-06 18:59:52.541587596 +0000 UTC m=+52.152913112 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls") pod "image-registry-74bc668d5c-lv5js" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d") : secret "image-registry-tls" not found
Jul 06 18:59:45.347139 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.347105    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:45.347304 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:45.347252    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 18:59:45.347342 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:45.347313    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert podName:a54d672b-7076-4068-92ab-5e1a4406abc9 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:53.34729751 +0000 UTC m=+52.958623026 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhr6r" (UID: "a54d672b-7076-4068-92ab-5e1a4406abc9") : secret "networking-console-plugin-cert" not found
Jul 06 18:59:45.439891 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.439863    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-q8fsz"]
Jul 06 18:59:45.459626 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.459603    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.462924 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.462907    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 18:59:45.463039 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.462907    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 18:59:45.464065 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.464042    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g47v9\""
Jul 06 18:59:45.464164 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.464077    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 18:59:45.464164 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.464079    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 18:59:45.470041 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.470023    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q8fsz"]
Jul 06 18:59:45.549065 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.549042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-crio-socket\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.549314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.549070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.549314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.549089    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9dm\" (UniqueName: \"kubernetes.io/projected/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-api-access-gx9dm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.549314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.549173    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.549314 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.549217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-data-volume\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.650055 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.650003    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-crio-socket\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.650055 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.650030    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.650055 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.650049    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9dm\" (UniqueName: \"kubernetes.io/projected/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-api-access-gx9dm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.650202 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.650109    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.650202 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:45.650128    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:59:45.650202 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.650141    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-data-volume\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.650202 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:45.650193    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:46.150177083 +0000 UTC m=+45.761502615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : configmap "kube-rbac-proxy" not found
Jul 06 18:59:45.650403 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.650216    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-crio-socket\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.650403 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:45.650283    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:45.650403 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:45.650349    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:46.150327725 +0000 UTC m=+45.761653264 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:45.650520 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.650435    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-data-volume\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:45.661481 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:45.661453    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9dm\" (UniqueName: \"kubernetes.io/projected/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-api-access-gx9dm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:46.153891 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.153861    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:46.154111 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.153923    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:46.154111 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:46.153977    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:59:46.154111 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:46.154036    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:47.154021753 +0000 UTC m=+46.765347269 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : configmap "kube-rbac-proxy" not found
Jul 06 18:59:46.154111 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:46.154037    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:46.154111 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:46.154083    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:47.15407253 +0000 UTC m=+46.765398073 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:46.409505 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.409423    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"]
Jul 06 18:59:46.432867 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.432843    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"
Jul 06 18:59:46.435774 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.435757    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 06 18:59:46.437155 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.437128    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-74lbr\""
Jul 06 18:59:46.437397 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.437376    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 06 18:59:46.438162 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.438139    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"]
Jul 06 18:59:46.456020 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.456001    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cqzz\" (UniqueName: \"kubernetes.io/projected/0f8e019f-79c6-4975-af71-7a33bf11665c-kube-api-access-7cqzz\") pod \"migrator-d57d58cc9-6qnch\" (UID: \"0f8e019f-79c6-4975-af71-7a33bf11665c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"
Jul 06 18:59:46.556215 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.556196    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cqzz\" (UniqueName: \"kubernetes.io/projected/0f8e019f-79c6-4975-af71-7a33bf11665c-kube-api-access-7cqzz\") pod \"migrator-d57d58cc9-6qnch\" (UID: \"0f8e019f-79c6-4975-af71-7a33bf11665c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"
Jul 06 18:59:46.565948 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.565929    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cqzz\" (UniqueName: \"kubernetes.io/projected/0f8e019f-79c6-4975-af71-7a33bf11665c-kube-api-access-7cqzz\") pod \"migrator-d57d58cc9-6qnch\" (UID: \"0f8e019f-79c6-4975-af71-7a33bf11665c\") " pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"
Jul 06 18:59:46.742732 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.742676    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"
Jul 06 18:59:46.901158 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:46.901129    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch"]
Jul 06 18:59:46.903867 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:46.903843    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8e019f_79c6_4975_af71_7a33bf11665c.slice/crio-f5b7d14f2f4a7adc337e00d54573a3449e9990043bf741f06651e57157451115 WatchSource:0}: Error finding container f5b7d14f2f4a7adc337e00d54573a3449e9990043bf741f06651e57157451115: Status 404 returned error can't find the container with id f5b7d14f2f4a7adc337e00d54573a3449e9990043bf741f06651e57157451115
Jul 06 18:59:47.161339 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:47.161305    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:47.161517 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:47.161407    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:47.161517 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:47.161473    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:47.161517 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:47.161495    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:59:47.161645 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:47.161531    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:49.161516165 +0000 UTC m=+48.772841682 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:47.161645 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:47.161545    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:49.16153866 +0000 UTC m=+48.772864176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : configmap "kube-rbac-proxy" not found
Jul 06 18:59:47.227246 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:47.227224    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twkgb_7f8025aa-247e-4f49-b65e-0830d654dc2b/dns-node-resolver/0.log"
Jul 06 18:59:47.237085 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:47.237063    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch" event={"ID":"0f8e019f-79c6-4975-af71-7a33bf11665c","Type":"ContainerStarted","Data":"f5b7d14f2f4a7adc337e00d54573a3449e9990043bf741f06651e57157451115"}
Jul 06 18:59:47.824281 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:47.824251    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxvwk_7c7a182e-597b-43d5-9c20-9e10ee7422c6/node-ca/0.log"
Jul 06 18:59:48.081311 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.081243    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-758f599c5-68zbn"]
Jul 06 18:59:48.097369 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.097347    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.099616 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.099596    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-68zbn"]
Jul 06 18:59:48.100464 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.100441    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 06 18:59:48.100764 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.100743    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 06 18:59:48.101621 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.101603    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-zs727\""
Jul 06 18:59:48.101729 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.101718    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 06 18:59:48.101948 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.101932    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 06 18:59:48.169772 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.169739    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wb65k\" (UniqueName: \"kubernetes.io/projected/5d48c491-a944-4d6c-9611-18e9aa3cdce6-kube-api-access-wb65k\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.169876 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.169846    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5d48c491-a944-4d6c-9611-18e9aa3cdce6-signing-cabundle\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.169976 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.169960    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5d48c491-a944-4d6c-9611-18e9aa3cdce6-signing-key\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.270882 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.270850    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wb65k\" (UniqueName: \"kubernetes.io/projected/5d48c491-a944-4d6c-9611-18e9aa3cdce6-kube-api-access-wb65k\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.271041 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.270942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5d48c491-a944-4d6c-9611-18e9aa3cdce6-signing-cabundle\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.271107 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.271047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5d48c491-a944-4d6c-9611-18e9aa3cdce6-signing-key\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.271577 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.271554    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5d48c491-a944-4d6c-9611-18e9aa3cdce6-signing-cabundle\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.273205 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.273186    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5d48c491-a944-4d6c-9611-18e9aa3cdce6-signing-key\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.279956 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.279929    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wb65k\" (UniqueName: \"kubernetes.io/projected/5d48c491-a944-4d6c-9611-18e9aa3cdce6-kube-api-access-wb65k\") pod \"service-ca-758f599c5-68zbn\" (UID: \"5d48c491-a944-4d6c-9611-18e9aa3cdce6\") " pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.408437 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.408374    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-758f599c5-68zbn"
Jul 06 18:59:48.559885 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.559858    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-758f599c5-68zbn"]
Jul 06 18:59:48.625274 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:48.625246    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2nz9k_d822dfd6-b5ec-4a8e-9fee-f5797dddac60/serve-healthcheck-canary/0.log"
Jul 06 18:59:48.656026 ip-10-0-142-165 kubenswrapper[2579]: W0706 18:59:48.656002    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d48c491_a944_4d6c_9611_18e9aa3cdce6.slice/crio-d9391665b8d65d92f6992ee2d81778172ed5a272edf75ba38c4231915ae9b080 WatchSource:0}: Error finding container d9391665b8d65d92f6992ee2d81778172ed5a272edf75ba38c4231915ae9b080: Status 404 returned error can't find the container with id d9391665b8d65d92f6992ee2d81778172ed5a272edf75ba38c4231915ae9b080
Jul 06 18:59:49.179843 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:49.179772    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:49.180228 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:49.179847    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:49.180228 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:49.179912    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:49.180228 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:49.179931    2579 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 18:59:49.180228 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:49.179984    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:53.179968564 +0000 UTC m=+52.791294080 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:49.180228 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:49.180013    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 18:59:53.180003229 +0000 UTC m=+52.791328749 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : configmap "kube-rbac-proxy" not found
Jul 06 18:59:49.243955 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:49.243919    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch" event={"ID":"0f8e019f-79c6-4975-af71-7a33bf11665c","Type":"ContainerStarted","Data":"3bfadc1b6aae0546f3aea2cec43c06d6b3405493f08fca5b9d65ed3bc558ce36"}
Jul 06 18:59:49.244110 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:49.243980    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch" event={"ID":"0f8e019f-79c6-4975-af71-7a33bf11665c","Type":"ContainerStarted","Data":"0f4acc87fac382d1277673e4c9d5c36a840d2eeb51bfc71054fa3b407a7125dc"}
Jul 06 18:59:49.245178 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:49.245147    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-68zbn" event={"ID":"5d48c491-a944-4d6c-9611-18e9aa3cdce6","Type":"ContainerStarted","Data":"d9391665b8d65d92f6992ee2d81778172ed5a272edf75ba38c4231915ae9b080"}
Jul 06 18:59:49.266257 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:49.266212    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-d57d58cc9-6qnch" podStartSLOduration=1.484429457 podStartE2EDuration="3.266198337s" totalImagesPullingTime="1.78176888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:49.264552339 +0000 UTC m=+48.875877877" watchObservedRunningTime="2026-07-06 18:59:49.266198337 +0000 UTC m=+48.877523875"
Jul 06 18:59:51.251762 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:51.251727    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-758f599c5-68zbn" event={"ID":"5d48c491-a944-4d6c-9611-18e9aa3cdce6","Type":"ContainerStarted","Data":"8e4125c0935dc09646b23c306ad6e1ef32d7cad9edeff90f3c4636db12271b9e"}
Jul 06 18:59:51.271221 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:51.271178    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-758f599c5-68zbn" podStartSLOduration=0.855356747 podStartE2EDuration="3.27116385s" totalImagesPullingTime="2.415807103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 18:59:51.270317679 +0000 UTC m=+50.881643235" watchObservedRunningTime="2026-07-06 18:59:51.27116385 +0000 UTC m=+50.882489443"
Jul 06 18:59:52.608524 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:52.608488    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 18:59:52.608524 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:52.608526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 18:59:52.608955 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:52.608645    2579 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 18:59:52.608955 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:52.608678    2579 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-74bc668d5c-lv5js: secret "image-registry-tls" not found
Jul 06 18:59:52.608955 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:52.608679    2579 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 18:59:52.608955 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:52.608733    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls podName:15c3ec3f-f235-4123-8af1-227508a8821d nodeName:}" failed. No retries permitted until 2026-07-06 19:00:08.608716182 +0000 UTC m=+68.220041701 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls") pod "image-registry-74bc668d5c-lv5js" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d") : secret "image-registry-tls" not found
Jul 06 18:59:52.608955 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:52.608747    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls podName:eda345d4-11ac-4c1e-9e3b-1607b869c7c2 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:08.608741351 +0000 UTC m=+68.220066867 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls") pod "dns-default-2ll9p" (UID: "eda345d4-11ac-4c1e-9e3b-1607b869c7c2") : secret "dns-default-metrics-tls" not found
Jul 06 18:59:53.213187 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:53.213148    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:53.213351 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:53.213264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:53.213351 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:53.213312    2579 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:53.213474 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:53.213385    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls podName:7bbdad23-8c7e-49c3-ab8a-de9caced9c31 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:01.213363036 +0000 UTC m=+60.824688554 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls") pod "insights-runtime-extractor-q8fsz" (UID: "7bbdad23-8c7e-49c3-ab8a-de9caced9c31") : secret "insights-runtime-extractor-tls" not found
Jul 06 18:59:53.213891 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:53.213872    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 18:59:53.415784 ip-10-0-142-165 kubenswrapper[2579]: I0706 18:59:53.415742    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 18:59:53.415966 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:53.415895    2579 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 06 18:59:53.416026 ip-10-0-142-165 kubenswrapper[2579]: E0706 18:59:53.415972    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert podName:a54d672b-7076-4068-92ab-5e1a4406abc9 nodeName:}" failed. No retries permitted until 2026-07-06 19:00:09.415946381 +0000 UTC m=+69.027271911 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert") pod "networking-console-plugin-6bffdf9778-xhr6r" (UID: "a54d672b-7076-4068-92ab-5e1a4406abc9") : secret "networking-console-plugin-cert" not found
Jul 06 19:00:01.277329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:01.277297    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 19:00:01.279439 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:01.279419    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7bbdad23-8c7e-49c3-ab8a-de9caced9c31-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-q8fsz\" (UID: \"7bbdad23-8c7e-49c3-ab8a-de9caced9c31\") " pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 19:00:01.371323 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:01.371295    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-g47v9\""
Jul 06 19:00:01.378796 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:01.378774    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-q8fsz"
Jul 06 19:00:01.503188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:01.503161    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-q8fsz"]
Jul 06 19:00:01.506431 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:01.506405    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bbdad23_8c7e_49c3_ab8a_de9caced9c31.slice/crio-dbe543b183dc2fcf1b31cfc43175e4bdaf83ccf87f1efe80d618def3241b97f3 WatchSource:0}: Error finding container dbe543b183dc2fcf1b31cfc43175e4bdaf83ccf87f1efe80d618def3241b97f3: Status 404 returned error can't find the container with id dbe543b183dc2fcf1b31cfc43175e4bdaf83ccf87f1efe80d618def3241b97f3
Jul 06 19:00:02.205650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:02.205625    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-zw8pk"
Jul 06 19:00:02.279870 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:02.279821    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8fsz" event={"ID":"7bbdad23-8c7e-49c3-ab8a-de9caced9c31","Type":"ContainerStarted","Data":"dbe543b183dc2fcf1b31cfc43175e4bdaf83ccf87f1efe80d618def3241b97f3"}
Jul 06 19:00:06.291814 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.291778    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8fsz" event={"ID":"7bbdad23-8c7e-49c3-ab8a-de9caced9c31","Type":"ContainerStarted","Data":"61171441588e1246901e1d57a5f98c496575736b0d826741ec6c60cbdbe47009"}
Jul 06 19:00:06.292148 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.291836    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8fsz" event={"ID":"7bbdad23-8c7e-49c3-ab8a-de9caced9c31","Type":"ContainerStarted","Data":"0a4342612e0508dbf6fe95e164cb48094bb230c9bad68a27e9a065f2c097aa00"}
Jul 06 19:00:06.727032 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.726989    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 19:00:06.727187 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.727084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 19:00:06.730162 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.730144    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 19:00:06.730240 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.730144    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 19:00:06.739466 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.739445    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/03aed337-be99-4849-94f5-0cdeeab4857f-original-pull-secret\") pod \"global-pull-secret-syncer-zmcn6\" (UID: \"03aed337-be99-4849-94f5-0cdeeab4857f\") " pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 19:00:06.739624 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.739607    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9209cc50-50be-47ab-b607-82201bf470a0-metrics-certs\") pod \"network-metrics-daemon-rspnw\" (UID: \"9209cc50-50be-47ab-b607-82201bf470a0\") " pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 19:00:06.828277 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.828244    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 19:00:06.831047 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.831029    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 19:00:06.841769 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.841747    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 19:00:06.852015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.851995    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkcfj\" (UniqueName: \"kubernetes.io/projected/9e0bcb48-8ed9-4de2-a59b-5db281e0059e-kube-api-access-mkcfj\") pod \"network-check-target-x44nm\" (UID: \"9e0bcb48-8ed9-4de2-a59b-5db281e0059e\") " pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 19:00:06.934554 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.934527    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zmcn6"
Jul 06 19:00:06.943148 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.943099    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-zn9xt\""
Jul 06 19:00:06.951399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:06.951250    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 19:00:07.034249 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.034214    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6kw7z\""
Jul 06 19:00:07.041429 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.041405    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rspnw"
Jul 06 19:00:07.090042 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.085795    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zmcn6"]
Jul 06 19:00:07.101556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.101530    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x44nm"]
Jul 06 19:00:07.104515 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:07.104361    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0bcb48_8ed9_4de2_a59b_5db281e0059e.slice/crio-0d594dbde5fce7f716a7eae8fcfd8d5e877befa073144a702b1c06b6d993d287 WatchSource:0}: Error finding container 0d594dbde5fce7f716a7eae8fcfd8d5e877befa073144a702b1c06b6d993d287: Status 404 returned error can't find the container with id 0d594dbde5fce7f716a7eae8fcfd8d5e877befa073144a702b1c06b6d993d287
Jul 06 19:00:07.191534 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.191503    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rspnw"]
Jul 06 19:00:07.194559 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:07.194513    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9209cc50_50be_47ab_b607_82201bf470a0.slice/crio-2ebe1e9b4e48d82ea8ce518dfedb2b4b2c9523c678f6fe1b36c57ff3fd643a7e WatchSource:0}: Error finding container 2ebe1e9b4e48d82ea8ce518dfedb2b4b2c9523c678f6fe1b36c57ff3fd643a7e: Status 404 returned error can't find the container with id 2ebe1e9b4e48d82ea8ce518dfedb2b4b2c9523c678f6fe1b36c57ff3fd643a7e
Jul 06 19:00:07.297173 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.297138    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rspnw" event={"ID":"9209cc50-50be-47ab-b607-82201bf470a0","Type":"ContainerStarted","Data":"2ebe1e9b4e48d82ea8ce518dfedb2b4b2c9523c678f6fe1b36c57ff3fd643a7e"}
Jul 06 19:00:07.298470 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.298436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zmcn6" event={"ID":"03aed337-be99-4849-94f5-0cdeeab4857f","Type":"ContainerStarted","Data":"641f0e5e06834443f758e9b7395fd92e782b32e2e4854b3c6c53e395328a158d"}
Jul 06 19:00:07.299628 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:07.299594    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x44nm" event={"ID":"9e0bcb48-8ed9-4de2-a59b-5db281e0059e","Type":"ContainerStarted","Data":"0d594dbde5fce7f716a7eae8fcfd8d5e877befa073144a702b1c06b6d993d287"}
Jul 06 19:00:08.645944 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.645893    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 19:00:08.645944 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.645946    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 19:00:08.648708 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.648680    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/eda345d4-11ac-4c1e-9e3b-1607b869c7c2-metrics-tls\") pod \"dns-default-2ll9p\" (UID: \"eda345d4-11ac-4c1e-9e3b-1607b869c7c2\") " pod="openshift-dns/dns-default-2ll9p"
Jul 06 19:00:08.648839 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.648792    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"image-registry-74bc668d5c-lv5js\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") " pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 19:00:08.859880 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.859847    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ch228\""
Jul 06 19:00:08.865532 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.865505    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2f7gj\""
Jul 06 19:00:08.867760 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.867616    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 19:00:08.873937 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:08.873879    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2ll9p"
Jul 06 19:00:09.453849 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:09.453811    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 19:00:09.457251 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:09.457217    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a54d672b-7076-4068-92ab-5e1a4406abc9-networking-console-plugin-cert\") pod \"networking-console-plugin-6bffdf9778-xhr6r\" (UID: \"a54d672b-7076-4068-92ab-5e1a4406abc9\") " pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 19:00:09.710842 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:09.710764    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-dll8m\""
Jul 06 19:00:09.718348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:09.718322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"
Jul 06 19:00:11.783390 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.783360    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74bc668d5c-lv5js"]
Jul 06 19:00:11.795251 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.793432    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"]
Jul 06 19:00:11.796475 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.796447    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.799994 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.799959    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 19:00:11.800104 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.800073    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 19:00:11.800369 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.800347    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 19:00:11.800603 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.800582    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 19:00:11.816257 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.816237    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"]
Jul 06 19:00:11.818035 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.818018    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:11.819895 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.819876    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"]
Jul 06 19:00:11.820702 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.820638    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-f5mt6\""
Jul 06 19:00:11.821153 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.821134    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 19:00:11.837154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.837131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"]
Jul 06 19:00:11.873855 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.873818    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e47655d-798d-4cd0-9849-d66f2141c78d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d5fdd8746-hcksh\" (UID: \"0e47655d-798d-4cd0-9849-d66f2141c78d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:11.874015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.873884    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tn2pl\" (UniqueName: \"kubernetes.io/projected/0e47655d-798d-4cd0-9849-d66f2141c78d-kube-api-access-tn2pl\") pod \"managed-serviceaccount-addon-agent-5d5fdd8746-hcksh\" (UID: \"0e47655d-798d-4cd0-9849-d66f2141c78d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:11.874015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.873957    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac014938-66e7-4fd0-be0e-33ec6c4656d4-tmp\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.874015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.873982    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ac014938-66e7-4fd0-be0e-33ec6c4656d4-klusterlet-config\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.874015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.874007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g6r6q\" (UniqueName: \"kubernetes.io/projected/ac014938-66e7-4fd0-be0e-33ec6c4656d4-kube-api-access-g6r6q\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.892572 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.892548    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"]
Jul 06 19:00:11.894571 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.894550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.921773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.921744    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"]
Jul 06 19:00:11.975176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92912c24-3423-4e41-bc36-49089b0ca461-installation-pull-secrets\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975194    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92912c24-3423-4e41-bc36-49089b0ca461-trusted-ca\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975216    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92912c24-3423-4e41-bc36-49089b0ca461-image-registry-private-configuration\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975280    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e47655d-798d-4cd0-9849-d66f2141c78d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d5fdd8746-hcksh\" (UID: \"0e47655d-798d-4cd0-9849-d66f2141c78d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:11.975502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975366    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92912c24-3423-4e41-bc36-49089b0ca461-registry-certificates\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975404    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-registry-tls\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975432    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-bound-sa-token\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975464    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tn2pl\" (UniqueName: \"kubernetes.io/projected/0e47655d-798d-4cd0-9849-d66f2141c78d-kube-api-access-tn2pl\") pod \"managed-serviceaccount-addon-agent-5d5fdd8746-hcksh\" (UID: \"0e47655d-798d-4cd0-9849-d66f2141c78d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:11.975502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975491    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mppsn\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-kube-api-access-mppsn\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975781 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975518    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92912c24-3423-4e41-bc36-49089b0ca461-ca-trust-extracted\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:11.975781 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975570    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac014938-66e7-4fd0-be0e-33ec6c4656d4-tmp\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.975781 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975598    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ac014938-66e7-4fd0-be0e-33ec6c4656d4-klusterlet-config\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.975781 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975627    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g6r6q\" (UniqueName: \"kubernetes.io/projected/ac014938-66e7-4fd0-be0e-33ec6c4656d4-kube-api-access-g6r6q\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.976000 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.975963    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac014938-66e7-4fd0-be0e-33ec6c4656d4-tmp\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.978197 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.978171    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0e47655d-798d-4cd0-9849-d66f2141c78d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5d5fdd8746-hcksh\" (UID: \"0e47655d-798d-4cd0-9849-d66f2141c78d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:11.978280 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.978224    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ac014938-66e7-4fd0-be0e-33ec6c4656d4-klusterlet-config\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.992171 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.992140    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g6r6q\" (UniqueName: \"kubernetes.io/projected/ac014938-66e7-4fd0-be0e-33ec6c4656d4-kube-api-access-g6r6q\") pod \"klusterlet-addon-workmgr-57bcf7646-crkkq\" (UID: \"ac014938-66e7-4fd0-be0e-33ec6c4656d4\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:11.992317 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:11.992294    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tn2pl\" (UniqueName: \"kubernetes.io/projected/0e47655d-798d-4cd0-9849-d66f2141c78d-kube-api-access-tn2pl\") pod \"managed-serviceaccount-addon-agent-5d5fdd8746-hcksh\" (UID: \"0e47655d-798d-4cd0-9849-d66f2141c78d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:12.076758 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.076726    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92912c24-3423-4e41-bc36-49089b0ca461-registry-certificates\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.076917 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.076773    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-registry-tls\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.076917 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.076798    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-bound-sa-token\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.076917 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.076827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mppsn\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-kube-api-access-mppsn\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.077063 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.077004    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92912c24-3423-4e41-bc36-49089b0ca461-ca-trust-extracted\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.077117 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.077086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92912c24-3423-4e41-bc36-49089b0ca461-installation-pull-secrets\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.077167 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.077128    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92912c24-3423-4e41-bc36-49089b0ca461-trusted-ca\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.077221 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.077163    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92912c24-3423-4e41-bc36-49089b0ca461-image-registry-private-configuration\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.077593 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.077538    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/92912c24-3423-4e41-bc36-49089b0ca461-ca-trust-extracted\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.078078 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.078030    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/92912c24-3423-4e41-bc36-49089b0ca461-registry-certificates\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.078283 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.078263    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/92912c24-3423-4e41-bc36-49089b0ca461-trusted-ca\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.079926 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.079908    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-registry-tls\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.080081 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.080060    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/92912c24-3423-4e41-bc36-49089b0ca461-image-registry-private-configuration\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.080176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.080156    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/92912c24-3423-4e41-bc36-49089b0ca461-installation-pull-secrets\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.092836 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.092814    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mppsn\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-kube-api-access-mppsn\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.093710 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.093690    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/92912c24-3423-4e41-bc36-49089b0ca461-bound-sa-token\") pod \"image-registry-56f9b5f9f8-rqzvd\" (UID: \"92912c24-3423-4e41-bc36-49089b0ca461\") " pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.107532 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.107511    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:12.134163 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.134139    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"
Jul 06 19:00:12.204802 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.204777    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:12.701421 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.701084    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2ll9p"]
Jul 06 19:00:12.764205 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.764130    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74bc668d5c-lv5js"]
Jul 06 19:00:12.768580 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:12.768530    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15c3ec3f_f235_4123_8af1_227508a8821d.slice/crio-78043ddb30a7bed1386d40cccb6885a82e1b31df0e766fa3852609cf1da6392d WatchSource:0}: Error finding container 78043ddb30a7bed1386d40cccb6885a82e1b31df0e766fa3852609cf1da6392d: Status 404 returned error can't find the container with id 78043ddb30a7bed1386d40cccb6885a82e1b31df0e766fa3852609cf1da6392d
Jul 06 19:00:12.787530 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.787119    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"]
Jul 06 19:00:12.793768 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:12.793464    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac014938_66e7_4fd0_be0e_33ec6c4656d4.slice/crio-59b2e2f591ed2a10efc074c5fcca48166113e28589ae922042809bffbe4db515 WatchSource:0}: Error finding container 59b2e2f591ed2a10efc074c5fcca48166113e28589ae922042809bffbe4db515: Status 404 returned error can't find the container with id 59b2e2f591ed2a10efc074c5fcca48166113e28589ae922042809bffbe4db515
Jul 06 19:00:12.873344 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:12.873284    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"]
Jul 06 19:00:12.877070 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:12.877035    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92912c24_3423_4e41_bc36_49089b0ca461.slice/crio-18f2c4004d8edc3313e896291be9a9a76e02a8f184e5871e498843dd0f3fbcbb WatchSource:0}: Error finding container 18f2c4004d8edc3313e896291be9a9a76e02a8f184e5871e498843dd0f3fbcbb: Status 404 returned error can't find the container with id 18f2c4004d8edc3313e896291be9a9a76e02a8f184e5871e498843dd0f3fbcbb
Jul 06 19:00:13.038077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.034518    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r"]
Jul 06 19:00:13.058292 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.058262    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh"]
Jul 06 19:00:13.062029 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:13.062007    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0e47655d_798d_4cd0_9849_d66f2141c78d.slice/crio-723e0b0ad6fce6ac52fd1d7ae3f24ef656310307a1b7b327e5a051de74e48152 WatchSource:0}: Error finding container 723e0b0ad6fce6ac52fd1d7ae3f24ef656310307a1b7b327e5a051de74e48152: Status 404 returned error can't find the container with id 723e0b0ad6fce6ac52fd1d7ae3f24ef656310307a1b7b327e5a051de74e48152
Jul 06 19:00:13.323965 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.323891    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r" event={"ID":"a54d672b-7076-4068-92ab-5e1a4406abc9","Type":"ContainerStarted","Data":"765d4053cb4232c1396ba3960387298d078223e36d4f15ff4d202f8e4f87e2cc"}
Jul 06 19:00:13.325030 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.325000    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ll9p" event={"ID":"eda345d4-11ac-4c1e-9e3b-1607b869c7c2","Type":"ContainerStarted","Data":"99772d788e16ac9c16f8ed52d59a0fb48ceed777125fab69a30cb793ac5556cb"}
Jul 06 19:00:13.326238 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.326213    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh" event={"ID":"0e47655d-798d-4cd0-9849-d66f2141c78d","Type":"ContainerStarted","Data":"723e0b0ad6fce6ac52fd1d7ae3f24ef656310307a1b7b327e5a051de74e48152"}
Jul 06 19:00:13.328238 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.328205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-q8fsz" event={"ID":"7bbdad23-8c7e-49c3-ab8a-de9caced9c31","Type":"ContainerStarted","Data":"0640d6a82a3e972d7f096ab93be092633e9b6d6b91b9d792541c0d7c90459e89"}
Jul 06 19:00:13.330039 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.330011    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rspnw" event={"ID":"9209cc50-50be-47ab-b607-82201bf470a0","Type":"ContainerStarted","Data":"3427516e86767d7978e2521fb3b8215fff2cd580b6797694611a281553cc5f5a"}
Jul 06 19:00:13.330155 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.330066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rspnw" event={"ID":"9209cc50-50be-47ab-b607-82201bf470a0","Type":"ContainerStarted","Data":"73997115e9763b7bb10d07ee97e7f4cd07be19c8d53fd15e3fc66ef66878e787"}
Jul 06 19:00:13.331384 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.331361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zmcn6" event={"ID":"03aed337-be99-4849-94f5-0cdeeab4857f","Type":"ContainerStarted","Data":"1613dcdf227b34afe6ad18cee6b659b84b16b154b96b5f3e0b0e7a5623526f02"}
Jul 06 19:00:13.332729 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.332709    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x44nm" event={"ID":"9e0bcb48-8ed9-4de2-a59b-5db281e0059e","Type":"ContainerStarted","Data":"0839f68f5ed33c1c8ef0d39da71afc1d566f98415d7bb67d264f7ce4af3bc62b"}
Jul 06 19:00:13.332802 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.332774    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 19:00:13.334001 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.333982    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd" event={"ID":"92912c24-3423-4e41-bc36-49089b0ca461","Type":"ContainerStarted","Data":"dbe4489ba1ad73b0191ca65abe6bac73835c9b4bdd13087c609760a8c1a185fd"}
Jul 06 19:00:13.334066 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.334018    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd" event={"ID":"92912c24-3423-4e41-bc36-49089b0ca461","Type":"ContainerStarted","Data":"18f2c4004d8edc3313e896291be9a9a76e02a8f184e5871e498843dd0f3fbcbb"}
Jul 06 19:00:13.334066 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.334038    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:13.335024 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.335008    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq" event={"ID":"ac014938-66e7-4fd0-be0e-33ec6c4656d4","Type":"ContainerStarted","Data":"59b2e2f591ed2a10efc074c5fcca48166113e28589ae922042809bffbe4db515"}
Jul 06 19:00:13.336275 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.336253    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js" event={"ID":"15c3ec3f-f235-4123-8af1-227508a8821d","Type":"ContainerStarted","Data":"e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b"}
Jul 06 19:00:13.336357 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.336293    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js" event={"ID":"15c3ec3f-f235-4123-8af1-227508a8821d","Type":"ContainerStarted","Data":"78043ddb30a7bed1386d40cccb6885a82e1b31df0e766fa3852609cf1da6392d"}
Jul 06 19:00:13.336357 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.336336    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 19:00:13.377820 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.377770    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-q8fsz" podStartSLOduration=17.493404817 podStartE2EDuration="28.377753991s" totalImagesPullingTime="10.884349174s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:45 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:13.37665047 +0000 UTC m=+72.987976007" watchObservedRunningTime="2026-07-06 19:00:13.377753991 +0000 UTC m=+72.989079507"
Jul 06 19:00:13.480521 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.480467    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js" podStartSLOduration=54.480451559 podStartE2EDuration="54.480451559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:13.416251665 +0000 UTC m=+73.027577206" watchObservedRunningTime="2026-07-06 19:00:13.480451559 +0000 UTC m=+73.091777096"
Jul 06 19:00:13.481399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.481351    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd" podStartSLOduration=2.481337189 podStartE2EDuration="2.481337189s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:13.479181709 +0000 UTC m=+73.090507248" watchObservedRunningTime="2026-07-06 19:00:13.481337189 +0000 UTC m=+73.092662721"
Jul 06 19:00:13.534993 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.534942    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rspnw" podStartSLOduration=67.242552895 podStartE2EDuration="1m12.534928123s" totalImagesPullingTime="5.292375228s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:13.533853838 +0000 UTC m=+73.145179376" watchObservedRunningTime="2026-07-06 19:00:13.534928123 +0000 UTC m=+73.146253661"
Jul 06 19:00:13.657812 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:13.657695    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zmcn6" podStartSLOduration=67.241727981 podStartE2EDuration="1m12.657673841s" totalImagesPullingTime="5.41594586s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:13.657414463 +0000 UTC m=+73.268740013" watchObservedRunningTime="2026-07-06 19:00:13.657673841 +0000 UTC m=+73.268999373"
Jul 06 19:00:15.349433 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:15.349392    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r" event={"ID":"a54d672b-7076-4068-92ab-5e1a4406abc9","Type":"ContainerStarted","Data":"fe621091c8c86ee8024d3ae8a526dc20db1766aec217b20c3fac36fc2e7a4288"}
Jul 06 19:00:15.351373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:15.351319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ll9p" event={"ID":"eda345d4-11ac-4c1e-9e3b-1607b869c7c2","Type":"ContainerStarted","Data":"d143ba3eb22ffd5e0257d1be75c8c4528eadab1df1dfac0569df1e05754da48a"}
Jul 06 19:00:15.371118 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:15.369445    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x44nm" podStartSLOduration=68.973094411 podStartE2EDuration="1m14.369425684s" totalImagesPullingTime="5.396331273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:13.734755912 +0000 UTC m=+73.346081450" watchObservedRunningTime="2026-07-06 19:00:15.369425684 +0000 UTC m=+74.980751212"
Jul 06 19:00:17.360739 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:17.360690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2ll9p" event={"ID":"eda345d4-11ac-4c1e-9e3b-1607b869c7c2","Type":"ContainerStarted","Data":"d4088dba25994339968950d63731c21947c357bbf54c7839f459a407a1c7d483"}
Jul 06 19:00:17.361175 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:17.360873    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2ll9p"
Jul 06 19:00:17.362413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:17.362388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh" event={"ID":"0e47655d-798d-4cd0-9849-d66f2141c78d","Type":"ContainerStarted","Data":"ff38cba87fd0723b9144bfc832673a42dad3fb7b3aadd40e4380c78d22bdbed6"}
Jul 06 19:00:17.398184 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:17.398139    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2ll9p" podStartSLOduration=38.932793229 podStartE2EDuration="41.398124075s" totalImagesPullingTime="2.465330846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:17.397718413 +0000 UTC m=+77.009043975" watchObservedRunningTime="2026-07-06 19:00:17.398124075 +0000 UTC m=+77.009449612"
Jul 06 19:00:17.398318 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:17.398232    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6bffdf9778-xhr6r" podStartSLOduration=38.262015043 podStartE2EDuration="40.398222268s" totalImagesPullingTime="2.136207225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 18:59:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:15.371005369 +0000 UTC m=+74.982330911" watchObservedRunningTime="2026-07-06 19:00:17.398222268 +0000 UTC m=+77.009547809"
Jul 06 19:00:17.459527 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:17.459463    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5d5fdd8746-hcksh" podStartSLOduration=3.014599715 podStartE2EDuration="6.459443103s" totalImagesPullingTime="3.444843388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:17.458379392 +0000 UTC m=+77.069704932" watchObservedRunningTime="2026-07-06 19:00:17.459443103 +0000 UTC m=+77.070768643"
Jul 06 19:00:19.371075 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:19.371043    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq" event={"ID":"ac014938-66e7-4fd0-be0e-33ec6c4656d4","Type":"ContainerStarted","Data":"e24a28f9df95c30e5163d72ad5735d5922591516564419939ec4220b4e8fac7f"}
Jul 06 19:00:19.371487 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:19.371253    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:19.372968 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:19.372949    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq"
Jul 06 19:00:19.393533 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:19.393488    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-57bcf7646-crkkq" podStartSLOduration=2.434465863 podStartE2EDuration="8.393475362s" totalImagesPullingTime="5.959009499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:19.392274621 +0000 UTC m=+79.003600158" watchObservedRunningTime="2026-07-06 19:00:19.393475362 +0000 UTC m=+79.004800879"
Jul 06 19:00:27.368644 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.368544    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2ll9p"
Jul 06 19:00:27.622765 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.622684    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cgm7z"]
Jul 06 19:00:27.626713 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.626691    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.632268 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.632252    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 19:00:27.636091 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.636075    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 19:00:27.636728 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.636706    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 19:00:27.636848 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.636333    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cr5b7\""
Jul 06 19:00:27.636903 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.636337    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 19:00:27.636952 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.636267    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 19:00:27.637025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.637011    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 19:00:27.805184 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805159    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-accelerators-collector-config\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805322 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-sys\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805322 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0675175b-5f36-4bab-8c2d-f50d50443d39-metrics-client-ca\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805322 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-root\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805322 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805306    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-textfile\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805479 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805332    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-tls\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805479 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805396    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvgjv\" (UniqueName: \"kubernetes.io/projected/0675175b-5f36-4bab-8c2d-f50d50443d39-kube-api-access-fvgjv\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805479 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805442    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-wtmp\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.805582 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.805477    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906648    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-accelerators-collector-config\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906711    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-sys\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906756    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0675175b-5f36-4bab-8c2d-f50d50443d39-metrics-client-ca\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906781    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-root\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-textfile\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906827    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-tls\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvgjv\" (UniqueName: \"kubernetes.io/projected/0675175b-5f36-4bab-8c2d-f50d50443d39-kube-api-access-fvgjv\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.906897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906899    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-wtmp\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.907361 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.906929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.907415 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.907362    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-root\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.907711 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.907685    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-sys\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.907936 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.907914    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-accelerators-collector-config\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.908037 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.907930    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-wtmp\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.908264 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.908244    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-textfile\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.908387 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.908363    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0675175b-5f36-4bab-8c2d-f50d50443d39-metrics-client-ca\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.909877 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.909856    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.911117 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.911097    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/0675175b-5f36-4bab-8c2d-f50d50443d39-node-exporter-tls\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.918376 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.918281    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvgjv\" (UniqueName: \"kubernetes.io/projected/0675175b-5f36-4bab-8c2d-f50d50443d39-kube-api-access-fvgjv\") pod \"node-exporter-cgm7z\" (UID: \"0675175b-5f36-4bab-8c2d-f50d50443d39\") " pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.938689 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:27.938651    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cgm7z"
Jul 06 19:00:27.949963 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:27.949943    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0675175b_5f36_4bab_8c2d_f50d50443d39.slice/crio-f344d42de666fd47b1e65125228477ed6e211cda4ec4d43104218d991740914f WatchSource:0}: Error finding container f344d42de666fd47b1e65125228477ed6e211cda4ec4d43104218d991740914f: Status 404 returned error can't find the container with id f344d42de666fd47b1e65125228477ed6e211cda4ec4d43104218d991740914f
Jul 06 19:00:28.401910 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:28.401881    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cgm7z" event={"ID":"0675175b-5f36-4bab-8c2d-f50d50443d39","Type":"ContainerStarted","Data":"f344d42de666fd47b1e65125228477ed6e211cda4ec4d43104218d991740914f"}
Jul 06 19:00:29.406459 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:29.406425    2579 generic.go:356] "Generic (PLEG): container finished" podID="0675175b-5f36-4bab-8c2d-f50d50443d39" containerID="91146db9e17667e814e96f796262468aeb6f32587fe2eb6d4a9fe99cd63808f2" exitCode=0
Jul 06 19:00:29.406804 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:29.406516    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cgm7z" event={"ID":"0675175b-5f36-4bab-8c2d-f50d50443d39","Type":"ContainerDied","Data":"91146db9e17667e814e96f796262468aeb6f32587fe2eb6d4a9fe99cd63808f2"}
Jul 06 19:00:30.412356 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:30.412319    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cgm7z" event={"ID":"0675175b-5f36-4bab-8c2d-f50d50443d39","Type":"ContainerStarted","Data":"3f3a56645addca417a8311a0b615327f08e7d8d5b031975422c3b5ca1695213c"}
Jul 06 19:00:30.412738 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:30.412403    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cgm7z" event={"ID":"0675175b-5f36-4bab-8c2d-f50d50443d39","Type":"ContainerStarted","Data":"1c15111362789365db4611804df3e26711ebeb8ce390831239a4900dfec18293"}
Jul 06 19:00:30.442738 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:30.442695    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cgm7z" podStartSLOduration=2.163482021 podStartE2EDuration="3.442680941s" totalImagesPullingTime="1.215277133s" totalInitContainerRuntime="63.921787ms" isStatefulPod=true podCreationTimestamp="2026-07-06 19:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:30.442284172 +0000 UTC m=+90.053609713" watchObservedRunningTime="2026-07-06 19:00:30.442680941 +0000 UTC m=+90.054006476"
Jul 06 19:00:32.209552 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:32.209517    2579 patch_prober.go:28] interesting pod/image-registry-56f9b5f9f8-rqzvd container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 19:00:32.209896 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:32.209570    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd" podUID="92912c24-3423-4e41-bc36-49089b0ca461" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 19:00:33.341813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:33.341775    2579 patch_prober.go:28] interesting pod/image-registry-74bc668d5c-lv5js container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 06 19:00:33.342161 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:33.341830    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js" podUID="15c3ec3f-f235-4123-8af1-227508a8821d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 19:00:34.347433 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:34.347404    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56f9b5f9f8-rqzvd"
Jul 06 19:00:38.355057 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.355010    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js" podUID="15c3ec3f-f235-4123-8af1-227508a8821d" containerName="registry" containerID="cri-o://e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b" gracePeriod=30
Jul 06 19:00:38.597608 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.597586    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 19:00:38.678639 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.678583    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15c3ec3f-f235-4123-8af1-227508a8821d-ca-trust-extracted\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.678639 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.678613    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-bound-sa-token\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.678639 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.678635    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.678858 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.678676    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-trusted-ca\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.678858 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.678705    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzbft\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-kube-api-access-dzbft\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.678858 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.678728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-installation-pull-secrets\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.678858 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.678766    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-registry-certificates\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.679046 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.679021    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-image-registry-private-configuration\") pod \"15c3ec3f-f235-4123-8af1-227508a8821d\" (UID: \"15c3ec3f-f235-4123-8af1-227508a8821d\") "
Jul 06 19:00:38.679099 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.679079    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:00:38.679329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.679149    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:00:38.679329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.679263    2579 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-registry-certificates\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:38.679329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.679281    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/15c3ec3f-f235-4123-8af1-227508a8821d-trusted-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:38.681408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.681362    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:38.681509 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.681465    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:38.681509 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.681487    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:00:38.681593 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.681571    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-kube-api-access-dzbft" (OuterVolumeSpecName: "kube-api-access-dzbft") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "kube-api-access-dzbft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:00:38.681631 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.681583    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:00:38.687484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.687459    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/15c3ec3f-f235-4123-8af1-227508a8821d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "15c3ec3f-f235-4123-8af1-227508a8821d" (UID: "15c3ec3f-f235-4123-8af1-227508a8821d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:00:38.779967 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.779947    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzbft\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-kube-api-access-dzbft\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:38.779967 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.779965    2579 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-installation-pull-secrets\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:38.780089 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.779976    2579 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/15c3ec3f-f235-4123-8af1-227508a8821d-image-registry-private-configuration\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:38.780089 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.779984    2579 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/15c3ec3f-f235-4123-8af1-227508a8821d-ca-trust-extracted\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:38.780089 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.779993    2579 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-bound-sa-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:38.780089 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:38.780001    2579 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/15c3ec3f-f235-4123-8af1-227508a8821d-registry-tls\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:00:39.440706 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.440676    2579 generic.go:356] "Generic (PLEG): container finished" podID="15c3ec3f-f235-4123-8af1-227508a8821d" containerID="e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b" exitCode=0
Jul 06 19:00:39.441070 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.440726    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js"
Jul 06 19:00:39.441070 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.440738    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js" event={"ID":"15c3ec3f-f235-4123-8af1-227508a8821d","Type":"ContainerDied","Data":"e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b"}
Jul 06 19:00:39.441070 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.440777    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74bc668d5c-lv5js" event={"ID":"15c3ec3f-f235-4123-8af1-227508a8821d","Type":"ContainerDied","Data":"78043ddb30a7bed1386d40cccb6885a82e1b31df0e766fa3852609cf1da6392d"}
Jul 06 19:00:39.441070 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.440794    2579 scope.go:117] "RemoveContainer" containerID="e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b"
Jul 06 19:00:39.459432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.459410    2579 scope.go:117] "RemoveContainer" containerID="e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b"
Jul 06 19:00:39.459687 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:00:39.459650    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b\": container with ID starting with e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b not found: ID does not exist" containerID="e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b"
Jul 06 19:00:39.459740 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.459695    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b"} err="failed to get container status \"e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b\": rpc error: code = NotFound desc = could not find container \"e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b\": container with ID starting with e509aa6b31ac7be235c13a1bf3dce16d76b476c51d19dd9867c738e7a86f8c0b not found: ID does not exist"
Jul 06 19:00:39.493770 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.493747    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-74bc668d5c-lv5js"]
Jul 06 19:00:39.506954 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:39.506936    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-74bc668d5c-lv5js"]
Jul 06 19:00:40.926408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:40.926377    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15c3ec3f-f235-4123-8af1-227508a8821d" path="/var/lib/kubelet/pods/15c3ec3f-f235-4123-8af1-227508a8821d/volumes"
Jul 06 19:00:44.346295 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:44.346262    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x44nm"
Jul 06 19:00:50.598495 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.598460    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6584bccf-5dhrh"]
Jul 06 19:00:50.598904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.598719    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15c3ec3f-f235-4123-8af1-227508a8821d" containerName="registry"
Jul 06 19:00:50.598904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.598730    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="15c3ec3f-f235-4123-8af1-227508a8821d" containerName="registry"
Jul 06 19:00:50.598904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.598788    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="15c3ec3f-f235-4123-8af1-227508a8821d" containerName="registry"
Jul 06 19:00:50.751360 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.751329    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6584bccf-5dhrh"]
Jul 06 19:00:50.751506 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.751416    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.754821 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.754788    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 19:00:50.755338 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.755316    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 19:00:50.755414 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.755323    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 19:00:50.758334 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.757949    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 19:00:50.758334 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.758192    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 19:00:50.758912 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.758883    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 19:00:50.759632 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.759616    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vzl4z\""
Jul 06 19:00:50.759735 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.759617    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 19:00:50.762838 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.762820    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 19:00:50.858087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.858036    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-console-config\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.858087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.858064    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-oauth-config\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.858087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.858082    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsjcj\" (UniqueName: \"kubernetes.io/projected/d4f2d77e-22a1-4601-b65d-b8912a859232-kube-api-access-nsjcj\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.858292 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.858107    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-oauth-serving-cert\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.858292 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.858171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-service-ca\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.858292 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.858242    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-trusted-ca-bundle\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.858292 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.858278    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-serving-cert\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.958824 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.958804    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-console-config\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.958937 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.958831    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-oauth-config\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.958937 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.958848    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsjcj\" (UniqueName: \"kubernetes.io/projected/d4f2d77e-22a1-4601-b65d-b8912a859232-kube-api-access-nsjcj\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.958937 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.958867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-oauth-serving-cert\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.959096 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.959062    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-service-ca\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.959150 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.959115    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-trusted-ca-bundle\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.959234 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.959212    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-serving-cert\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.959612 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.959583    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-oauth-serving-cert\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.959721 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.959642    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-console-config\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.959922 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.959899    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-service-ca\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.960014 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.959978    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-trusted-ca-bundle\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.961244 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.961217    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-oauth-config\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.961521 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.961503    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-serving-cert\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:50.967844 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:50.967824    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsjcj\" (UniqueName: \"kubernetes.io/projected/d4f2d77e-22a1-4601-b65d-b8912a859232-kube-api-access-nsjcj\") pod \"console-6584bccf-5dhrh\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") " pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:51.060763 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:51.060743    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:00:51.196957 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:51.196923    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6584bccf-5dhrh"]
Jul 06 19:00:51.199686 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:00:51.199644    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4f2d77e_22a1_4601_b65d_b8912a859232.slice/crio-630d5bab3772cd025fa07d6caca9c59cd2fd3077d1d14e6ab64109cbec52d12c WatchSource:0}: Error finding container 630d5bab3772cd025fa07d6caca9c59cd2fd3077d1d14e6ab64109cbec52d12c: Status 404 returned error can't find the container with id 630d5bab3772cd025fa07d6caca9c59cd2fd3077d1d14e6ab64109cbec52d12c
Jul 06 19:00:51.483441 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:51.483348    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6584bccf-5dhrh" event={"ID":"d4f2d77e-22a1-4601-b65d-b8912a859232","Type":"ContainerStarted","Data":"630d5bab3772cd025fa07d6caca9c59cd2fd3077d1d14e6ab64109cbec52d12c"}
Jul 06 19:00:55.500646 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:55.500613    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6584bccf-5dhrh" event={"ID":"d4f2d77e-22a1-4601-b65d-b8912a859232","Type":"ContainerStarted","Data":"b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a"}
Jul 06 19:00:55.521074 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:00:55.521023    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6584bccf-5dhrh" podStartSLOduration=1.9698584829999999 podStartE2EDuration="5.52100535s" totalImagesPullingTime="3.551146867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:00:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:00:55.519998629 +0000 UTC m=+115.131324168" watchObservedRunningTime="2026-07-06 19:00:55.52100535 +0000 UTC m=+115.132330888"
Jul 06 19:01:01.061738 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:01:01.061618    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:01:01.061738 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:01:01.061719    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:01:01.066494 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:01:01.066471    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:01:01.527240 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:01:01.527165    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:02:04.294218 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:04.294144    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6584bccf-5dhrh"]
Jul 06 19:02:29.313550 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.313502    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-6584bccf-5dhrh" podUID="d4f2d77e-22a1-4601-b65d-b8912a859232" containerName="console" containerID="cri-o://b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a" gracePeriod=15
Jul 06 19:02:29.542413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.542393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:02:29.640651 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.640595    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-trusted-ca-bundle\") pod \"d4f2d77e-22a1-4601-b65d-b8912a859232\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") "
Jul 06 19:02:29.640651 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.640636    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-console-config\") pod \"d4f2d77e-22a1-4601-b65d-b8912a859232\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") "
Jul 06 19:02:29.640841 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.640671    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-oauth-serving-cert\") pod \"d4f2d77e-22a1-4601-b65d-b8912a859232\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") "
Jul 06 19:02:29.640841 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.640743    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-oauth-config\") pod \"d4f2d77e-22a1-4601-b65d-b8912a859232\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") "
Jul 06 19:02:29.640841 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.640765    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-serving-cert\") pod \"d4f2d77e-22a1-4601-b65d-b8912a859232\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") "
Jul 06 19:02:29.640841 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.640792    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsjcj\" (UniqueName: \"kubernetes.io/projected/d4f2d77e-22a1-4601-b65d-b8912a859232-kube-api-access-nsjcj\") pod \"d4f2d77e-22a1-4601-b65d-b8912a859232\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") "
Jul 06 19:02:29.640841 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.640817    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-service-ca\") pod \"d4f2d77e-22a1-4601-b65d-b8912a859232\" (UID: \"d4f2d77e-22a1-4601-b65d-b8912a859232\") "
Jul 06 19:02:29.641127 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.641102    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-console-config" (OuterVolumeSpecName: "console-config") pod "d4f2d77e-22a1-4601-b65d-b8912a859232" (UID: "d4f2d77e-22a1-4601-b65d-b8912a859232"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:02:29.641243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.641108    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d4f2d77e-22a1-4601-b65d-b8912a859232" (UID: "d4f2d77e-22a1-4601-b65d-b8912a859232"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:02:29.641356 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.641256    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d4f2d77e-22a1-4601-b65d-b8912a859232" (UID: "d4f2d77e-22a1-4601-b65d-b8912a859232"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:02:29.641424 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.641355    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-service-ca" (OuterVolumeSpecName: "service-ca") pod "d4f2d77e-22a1-4601-b65d-b8912a859232" (UID: "d4f2d77e-22a1-4601-b65d-b8912a859232"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:02:29.643073 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.643048    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d4f2d77e-22a1-4601-b65d-b8912a859232" (UID: "d4f2d77e-22a1-4601-b65d-b8912a859232"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:02:29.643151 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.643066    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d4f2d77e-22a1-4601-b65d-b8912a859232" (UID: "d4f2d77e-22a1-4601-b65d-b8912a859232"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:02:29.643151 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.643108    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4f2d77e-22a1-4601-b65d-b8912a859232-kube-api-access-nsjcj" (OuterVolumeSpecName: "kube-api-access-nsjcj") pod "d4f2d77e-22a1-4601-b65d-b8912a859232" (UID: "d4f2d77e-22a1-4601-b65d-b8912a859232"). InnerVolumeSpecName "kube-api-access-nsjcj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:02:29.741725 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.741700    2579 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-console-config\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:02:29.741725 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.741721    2579 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-oauth-serving-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:02:29.741843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.741731    2579 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-oauth-config\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:02:29.741843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.741740    2579 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d4f2d77e-22a1-4601-b65d-b8912a859232-console-serving-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:02:29.741843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.741748    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsjcj\" (UniqueName: \"kubernetes.io/projected/d4f2d77e-22a1-4601-b65d-b8912a859232-kube-api-access-nsjcj\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:02:29.741843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.741757    2579 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-service-ca\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:02:29.741843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.741764    2579 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d4f2d77e-22a1-4601-b65d-b8912a859232-trusted-ca-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:02:29.808985 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.808955    2579 generic.go:356] "Generic (PLEG): container finished" podID="d4f2d77e-22a1-4601-b65d-b8912a859232" containerID="b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a" exitCode=0
Jul 06 19:02:29.809103 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.809023    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6584bccf-5dhrh"
Jul 06 19:02:29.809103 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.809046    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6584bccf-5dhrh" event={"ID":"d4f2d77e-22a1-4601-b65d-b8912a859232","Type":"ContainerDied","Data":"b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a"}
Jul 06 19:02:29.809285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.809107    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6584bccf-5dhrh" event={"ID":"d4f2d77e-22a1-4601-b65d-b8912a859232","Type":"ContainerDied","Data":"630d5bab3772cd025fa07d6caca9c59cd2fd3077d1d14e6ab64109cbec52d12c"}
Jul 06 19:02:29.809285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.809133    2579 scope.go:117] "RemoveContainer" containerID="b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a"
Jul 06 19:02:29.817207 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.817189    2579 scope.go:117] "RemoveContainer" containerID="b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a"
Jul 06 19:02:29.817461 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:02:29.817440    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a\": container with ID starting with b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a not found: ID does not exist" containerID="b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a"
Jul 06 19:02:29.817509 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.817471    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a"} err="failed to get container status \"b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a\": rpc error: code = NotFound desc = could not find container \"b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a\": container with ID starting with b82ca976bbec9a4414c360fdae7387ec6030b59519189f5e6b73252858bd6e8a not found: ID does not exist"
Jul 06 19:02:29.835386 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.835366    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6584bccf-5dhrh"]
Jul 06 19:02:29.842348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:29.842327    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6584bccf-5dhrh"]
Jul 06 19:02:30.926005 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:02:30.925972    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4f2d77e-22a1-4601-b65d-b8912a859232" path="/var/lib/kubelet/pods/d4f2d77e-22a1-4601-b65d-b8912a859232/volumes"
Jul 06 19:04:00.809494 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:00.809461    2579 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 19:04:20.441051 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.441011    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"]
Jul 06 19:04:20.443462 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.441365    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4f2d77e-22a1-4601-b65d-b8912a859232" containerName="console"
Jul 06 19:04:20.443462 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.441382    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4f2d77e-22a1-4601-b65d-b8912a859232" containerName="console"
Jul 06 19:04:20.443462 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.441439    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4f2d77e-22a1-4601-b65d-b8912a859232" containerName="console"
Jul 06 19:04:20.444333 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.444313    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.446850 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.446825    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 19:04:20.447213 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.447199    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6t7xr\""
Jul 06 19:04:20.447890 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.447874    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 19:04:20.460863 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.460839    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"]
Jul 06 19:04:20.575889 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.575852    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x292\" (UniqueName: \"kubernetes.io/projected/261837aa-8f7a-487b-8dcb-f27339387635-kube-api-access-8x292\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.576071 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.575908    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.576071 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.575936    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.676562 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.676524    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.676562 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.676564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.676771 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.676620    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x292\" (UniqueName: \"kubernetes.io/projected/261837aa-8f7a-487b-8dcb-f27339387635-kube-api-access-8x292\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.677023 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.676997    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.677096 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.677006    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.686892 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.686869    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x292\" (UniqueName: \"kubernetes.io/projected/261837aa-8f7a-487b-8dcb-f27339387635-kube-api-access-8x292\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.757306 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.757219    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:20.886975 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.886949    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"]
Jul 06 19:04:20.888830 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:04:20.888807    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod261837aa_8f7a_487b_8dcb_f27339387635.slice/crio-b18b574f7e2a10cb37d52eeafd9a643eaa1ad58e50ee24c10bf7fd6a8e0120f7 WatchSource:0}: Error finding container b18b574f7e2a10cb37d52eeafd9a643eaa1ad58e50ee24c10bf7fd6a8e0120f7: Status 404 returned error can't find the container with id b18b574f7e2a10cb37d52eeafd9a643eaa1ad58e50ee24c10bf7fd6a8e0120f7
Jul 06 19:04:20.891024 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:20.891009    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:04:21.159146 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:21.159110    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8" event={"ID":"261837aa-8f7a-487b-8dcb-f27339387635","Type":"ContainerStarted","Data":"b18b574f7e2a10cb37d52eeafd9a643eaa1ad58e50ee24c10bf7fd6a8e0120f7"}
Jul 06 19:04:31.193021 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:31.192984    2579 generic.go:356] "Generic (PLEG): container finished" podID="261837aa-8f7a-487b-8dcb-f27339387635" containerID="410bb32488c2b01ada3be8d329dad23224e18c031406355017931710cab53a36" exitCode=0
Jul 06 19:04:31.193389 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:31.193041    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8" event={"ID":"261837aa-8f7a-487b-8dcb-f27339387635","Type":"ContainerDied","Data":"410bb32488c2b01ada3be8d329dad23224e18c031406355017931710cab53a36"}
Jul 06 19:04:34.205547 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:34.205510    2579 generic.go:356] "Generic (PLEG): container finished" podID="261837aa-8f7a-487b-8dcb-f27339387635" containerID="6ce112b6c8e5366227d7c34b469055b5a1674962b94993366cc47ef7df9eece9" exitCode=0
Jul 06 19:04:34.205973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:34.205566    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8" event={"ID":"261837aa-8f7a-487b-8dcb-f27339387635","Type":"ContainerDied","Data":"6ce112b6c8e5366227d7c34b469055b5a1674962b94993366cc47ef7df9eece9"}
Jul 06 19:04:44.240003 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:44.239970    2579 generic.go:356] "Generic (PLEG): container finished" podID="261837aa-8f7a-487b-8dcb-f27339387635" containerID="9ebf94afbb2b84078de4ff77d17de1e942f6e94ceca129af4d5118b7740f3377" exitCode=0
Jul 06 19:04:44.240434 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:44.240012    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8" event={"ID":"261837aa-8f7a-487b-8dcb-f27339387635","Type":"ContainerDied","Data":"9ebf94afbb2b84078de4ff77d17de1e942f6e94ceca129af4d5118b7740f3377"}
Jul 06 19:04:45.363265 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.363242    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:45.450362 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.450335    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x292\" (UniqueName: \"kubernetes.io/projected/261837aa-8f7a-487b-8dcb-f27339387635-kube-api-access-8x292\") pod \"261837aa-8f7a-487b-8dcb-f27339387635\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") "
Jul 06 19:04:45.450501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.450378    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-bundle\") pod \"261837aa-8f7a-487b-8dcb-f27339387635\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") "
Jul 06 19:04:45.450501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.450396    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-util\") pod \"261837aa-8f7a-487b-8dcb-f27339387635\" (UID: \"261837aa-8f7a-487b-8dcb-f27339387635\") "
Jul 06 19:04:45.450999 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.450971    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-bundle" (OuterVolumeSpecName: "bundle") pod "261837aa-8f7a-487b-8dcb-f27339387635" (UID: "261837aa-8f7a-487b-8dcb-f27339387635"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:04:45.452394 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.452374    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/261837aa-8f7a-487b-8dcb-f27339387635-kube-api-access-8x292" (OuterVolumeSpecName: "kube-api-access-8x292") pod "261837aa-8f7a-487b-8dcb-f27339387635" (UID: "261837aa-8f7a-487b-8dcb-f27339387635"). InnerVolumeSpecName "kube-api-access-8x292". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:04:45.454743 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.454717    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-util" (OuterVolumeSpecName: "util") pod "261837aa-8f7a-487b-8dcb-f27339387635" (UID: "261837aa-8f7a-487b-8dcb-f27339387635"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:04:45.551752 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.551725    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x292\" (UniqueName: \"kubernetes.io/projected/261837aa-8f7a-487b-8dcb-f27339387635-kube-api-access-8x292\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:04:45.551752 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.551755    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:04:45.551883 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:45.551764    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/261837aa-8f7a-487b-8dcb-f27339387635-util\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:04:46.250569 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:46.250527    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8" event={"ID":"261837aa-8f7a-487b-8dcb-f27339387635","Type":"ContainerDied","Data":"b18b574f7e2a10cb37d52eeafd9a643eaa1ad58e50ee24c10bf7fd6a8e0120f7"}
Jul 06 19:04:46.250797 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:46.250592    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b18b574f7e2a10cb37d52eeafd9a643eaa1ad58e50ee24c10bf7fd6a8e0120f7"
Jul 06 19:04:46.250797 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:46.250538    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675407c9z8"
Jul 06 19:04:54.451058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.450972    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"]
Jul 06 19:04:54.451501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.451315    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261837aa-8f7a-487b-8dcb-f27339387635" containerName="util"
Jul 06 19:04:54.451501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.451333    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="261837aa-8f7a-487b-8dcb-f27339387635" containerName="util"
Jul 06 19:04:54.451501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.451345    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261837aa-8f7a-487b-8dcb-f27339387635" containerName="pull"
Jul 06 19:04:54.451501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.451355    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="261837aa-8f7a-487b-8dcb-f27339387635" containerName="pull"
Jul 06 19:04:54.451501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.451376    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="261837aa-8f7a-487b-8dcb-f27339387635" containerName="extract"
Jul 06 19:04:54.451501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.451383    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="261837aa-8f7a-487b-8dcb-f27339387635" containerName="extract"
Jul 06 19:04:54.451501 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.451444    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="261837aa-8f7a-487b-8dcb-f27339387635" containerName="extract"
Jul 06 19:04:54.453368 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.453348    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.456394 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.456372    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:04:54.456582 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.456567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-f4djr\""
Jul 06 19:04:54.457255 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.457240    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 06 19:04:54.508661 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.508641    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70080073-392f-4890-8314-d5dccd179fc1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bkxl9\" (UID: \"70080073-392f-4890-8314-d5dccd179fc1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.508745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.508693    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mk4z\" (UniqueName: \"kubernetes.io/projected/70080073-392f-4890-8314-d5dccd179fc1-kube-api-access-8mk4z\") pod \"cert-manager-operator-controller-manager-55c444bf86-bkxl9\" (UID: \"70080073-392f-4890-8314-d5dccd179fc1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.511586 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.511564    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"]
Jul 06 19:04:54.609795 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.609774    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70080073-392f-4890-8314-d5dccd179fc1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bkxl9\" (UID: \"70080073-392f-4890-8314-d5dccd179fc1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.609896 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.609809    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8mk4z\" (UniqueName: \"kubernetes.io/projected/70080073-392f-4890-8314-d5dccd179fc1-kube-api-access-8mk4z\") pod \"cert-manager-operator-controller-manager-55c444bf86-bkxl9\" (UID: \"70080073-392f-4890-8314-d5dccd179fc1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.610144 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.610123    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70080073-392f-4890-8314-d5dccd179fc1-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-bkxl9\" (UID: \"70080073-392f-4890-8314-d5dccd179fc1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.631664 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.631640    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mk4z\" (UniqueName: \"kubernetes.io/projected/70080073-392f-4890-8314-d5dccd179fc1-kube-api-access-8mk4z\") pod \"cert-manager-operator-controller-manager-55c444bf86-bkxl9\" (UID: \"70080073-392f-4890-8314-d5dccd179fc1\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.762461 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.762393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"
Jul 06 19:04:54.898803 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:54.898768    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9"]
Jul 06 19:04:54.901819 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:04:54.901789    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70080073_392f_4890_8314_d5dccd179fc1.slice/crio-17098327a3359179beede91ae5e2327d1c2c5c17119208980b0401dd47e46f34 WatchSource:0}: Error finding container 17098327a3359179beede91ae5e2327d1c2c5c17119208980b0401dd47e46f34: Status 404 returned error can't find the container with id 17098327a3359179beede91ae5e2327d1c2c5c17119208980b0401dd47e46f34
Jul 06 19:04:55.281177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:55.281139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9" event={"ID":"70080073-392f-4890-8314-d5dccd179fc1","Type":"ContainerStarted","Data":"17098327a3359179beede91ae5e2327d1c2c5c17119208980b0401dd47e46f34"}
Jul 06 19:04:58.294805 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:58.294774    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9" event={"ID":"70080073-392f-4890-8314-d5dccd179fc1","Type":"ContainerStarted","Data":"6a28c87b5f2d1aae06ac9a0b4e30295f0d01260f5b67b18e580b76444eec82ab"}
Jul 06 19:04:58.344325 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:58.344281    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-bkxl9" podStartSLOduration=1.86858963 podStartE2EDuration="4.344265487s" totalImagesPullingTime="2.475675857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:04:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:04:58.342601599 +0000 UTC m=+357.953927136" watchObservedRunningTime="2026-07-06 19:04:58.344265487 +0000 UTC m=+357.955591025"
Jul 06 19:04:59.153559 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.153528    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"]
Jul 06 19:04:59.155946 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.155929    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.161364 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.161336    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6t7xr\""
Jul 06 19:04:59.161489 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.161384    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 19:04:59.162258 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.162239    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 19:04:59.195198 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.195174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"]
Jul 06 19:04:59.240204 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.240179    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqrm9\" (UniqueName: \"kubernetes.io/projected/999cdd90-b192-47bd-86e6-24f69ff71610-kube-api-access-vqrm9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.240306 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.240228    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.240306 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.240246    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.341116 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.341086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.341505 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.341122    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.341505 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.341168    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqrm9\" (UniqueName: \"kubernetes.io/projected/999cdd90-b192-47bd-86e6-24f69ff71610-kube-api-access-vqrm9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.341587 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.341543    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.341587 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.341554    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.373139 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.373116    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqrm9\" (UniqueName: \"kubernetes.io/projected/999cdd90-b192-47bd-86e6-24f69ff71610-kube-api-access-vqrm9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.464340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.464289    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:04:59.595376 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:04:59.595347    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"]
Jul 06 19:04:59.598071 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:04:59.598041    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999cdd90_b192_47bd_86e6_24f69ff71610.slice/crio-eeaf01c180a36b952c279fc9ece7215d2e3dcd5f492d09c94de34add46170755 WatchSource:0}: Error finding container eeaf01c180a36b952c279fc9ece7215d2e3dcd5f492d09c94de34add46170755: Status 404 returned error can't find the container with id eeaf01c180a36b952c279fc9ece7215d2e3dcd5f492d09c94de34add46170755
Jul 06 19:05:00.303625 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:00.303591    2579 generic.go:356] "Generic (PLEG): container finished" podID="999cdd90-b192-47bd-86e6-24f69ff71610" containerID="5c3b71b60a6f7504e0a2377f708594eab2a315532cc3100ed8cf5949b5428fc0" exitCode=0
Jul 06 19:05:00.303984 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:00.303636    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn" event={"ID":"999cdd90-b192-47bd-86e6-24f69ff71610","Type":"ContainerDied","Data":"5c3b71b60a6f7504e0a2377f708594eab2a315532cc3100ed8cf5949b5428fc0"}
Jul 06 19:05:00.303984 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:00.303716    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn" event={"ID":"999cdd90-b192-47bd-86e6-24f69ff71610","Type":"ContainerStarted","Data":"eeaf01c180a36b952c279fc9ece7215d2e3dcd5f492d09c94de34add46170755"}
Jul 06 19:05:03.319400 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:03.319362    2579 generic.go:356] "Generic (PLEG): container finished" podID="999cdd90-b192-47bd-86e6-24f69ff71610" containerID="ffa9dd103961ebf31d576e240d0126c7012ba066f4039ff9c8d55b55666a8def" exitCode=0
Jul 06 19:05:03.319771 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:03.319439    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn" event={"ID":"999cdd90-b192-47bd-86e6-24f69ff71610","Type":"ContainerDied","Data":"ffa9dd103961ebf31d576e240d0126c7012ba066f4039ff9c8d55b55666a8def"}
Jul 06 19:05:04.325316 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:04.325280    2579 generic.go:356] "Generic (PLEG): container finished" podID="999cdd90-b192-47bd-86e6-24f69ff71610" containerID="697d5af5052ff70dd6bf22bda24e9368231cc95f0c07a7d11ebb5ab7dfd4c4db" exitCode=0
Jul 06 19:05:04.325736 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:04.325362    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn" event={"ID":"999cdd90-b192-47bd-86e6-24f69ff71610","Type":"ContainerDied","Data":"697d5af5052ff70dd6bf22bda24e9368231cc95f0c07a7d11ebb5ab7dfd4c4db"}
Jul 06 19:05:05.446904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.446875    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:05:05.486948 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.486916    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vqrm9\" (UniqueName: \"kubernetes.io/projected/999cdd90-b192-47bd-86e6-24f69ff71610-kube-api-access-vqrm9\") pod \"999cdd90-b192-47bd-86e6-24f69ff71610\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") "
Jul 06 19:05:05.487102 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.486980    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-util\") pod \"999cdd90-b192-47bd-86e6-24f69ff71610\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") "
Jul 06 19:05:05.487102 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.487019    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-bundle\") pod \"999cdd90-b192-47bd-86e6-24f69ff71610\" (UID: \"999cdd90-b192-47bd-86e6-24f69ff71610\") "
Jul 06 19:05:05.487377 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.487352    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-bundle" (OuterVolumeSpecName: "bundle") pod "999cdd90-b192-47bd-86e6-24f69ff71610" (UID: "999cdd90-b192-47bd-86e6-24f69ff71610"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:05:05.489061 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.489037    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/999cdd90-b192-47bd-86e6-24f69ff71610-kube-api-access-vqrm9" (OuterVolumeSpecName: "kube-api-access-vqrm9") pod "999cdd90-b192-47bd-86e6-24f69ff71610" (UID: "999cdd90-b192-47bd-86e6-24f69ff71610"). InnerVolumeSpecName "kube-api-access-vqrm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:05:05.491937 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.491897    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-util" (OuterVolumeSpecName: "util") pod "999cdd90-b192-47bd-86e6-24f69ff71610" (UID: "999cdd90-b192-47bd-86e6-24f69ff71610"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:05:05.588188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.588108    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-util\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:05:05.588188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.588137    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/999cdd90-b192-47bd-86e6-24f69ff71610-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:05:05.588188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:05.588146    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqrm9\" (UniqueName: \"kubernetes.io/projected/999cdd90-b192-47bd-86e6-24f69ff71610-kube-api-access-vqrm9\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:05:06.334896 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:06.334871    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn"
Jul 06 19:05:06.335057 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:06.334865    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fh25cn" event={"ID":"999cdd90-b192-47bd-86e6-24f69ff71610","Type":"ContainerDied","Data":"eeaf01c180a36b952c279fc9ece7215d2e3dcd5f492d09c94de34add46170755"}
Jul 06 19:05:06.335057 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:06.334985    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eeaf01c180a36b952c279fc9ece7215d2e3dcd5f492d09c94de34add46170755"
Jul 06 19:05:11.402832 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.402796    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j48g9"]
Jul 06 19:05:11.403176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.403047    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999cdd90-b192-47bd-86e6-24f69ff71610" containerName="pull"
Jul 06 19:05:11.403176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.403059    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="999cdd90-b192-47bd-86e6-24f69ff71610" containerName="pull"
Jul 06 19:05:11.403176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.403070    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999cdd90-b192-47bd-86e6-24f69ff71610" containerName="util"
Jul 06 19:05:11.403176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.403075    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="999cdd90-b192-47bd-86e6-24f69ff71610" containerName="util"
Jul 06 19:05:11.403176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.403081    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999cdd90-b192-47bd-86e6-24f69ff71610" containerName="extract"
Jul 06 19:05:11.403176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.403085    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="999cdd90-b192-47bd-86e6-24f69ff71610" containerName="extract"
Jul 06 19:05:11.403176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.403138    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="999cdd90-b192-47bd-86e6-24f69ff71610" containerName="extract"
Jul 06 19:05:11.405276 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.405257    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.412162 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.412147    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 19:05:11.412539 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.412519    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 19:05:11.418331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.418310    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j48g9"]
Jul 06 19:05:11.427016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.426995    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpx2j\" (UniqueName: \"kubernetes.io/projected/1541f6e5-7700-4afb-8e59-7d03662da976-kube-api-access-fpx2j\") pod \"cert-manager-55b4d954d5-j48g9\" (UID: \"1541f6e5-7700-4afb-8e59-7d03662da976\") " pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.427100 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.427042    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1541f6e5-7700-4afb-8e59-7d03662da976-bound-sa-token\") pod \"cert-manager-55b4d954d5-j48g9\" (UID: \"1541f6e5-7700-4afb-8e59-7d03662da976\") " pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.528091 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.528061    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpx2j\" (UniqueName: \"kubernetes.io/projected/1541f6e5-7700-4afb-8e59-7d03662da976-kube-api-access-fpx2j\") pod \"cert-manager-55b4d954d5-j48g9\" (UID: \"1541f6e5-7700-4afb-8e59-7d03662da976\") " pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.528236 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.528111    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1541f6e5-7700-4afb-8e59-7d03662da976-bound-sa-token\") pod \"cert-manager-55b4d954d5-j48g9\" (UID: \"1541f6e5-7700-4afb-8e59-7d03662da976\") " pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.542626 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.542600    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1541f6e5-7700-4afb-8e59-7d03662da976-bound-sa-token\") pod \"cert-manager-55b4d954d5-j48g9\" (UID: \"1541f6e5-7700-4afb-8e59-7d03662da976\") " pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.544234 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.544217    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpx2j\" (UniqueName: \"kubernetes.io/projected/1541f6e5-7700-4afb-8e59-7d03662da976-kube-api-access-fpx2j\") pod \"cert-manager-55b4d954d5-j48g9\" (UID: \"1541f6e5-7700-4afb-8e59-7d03662da976\") " pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.627233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.627202    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"]
Jul 06 19:05:11.632530 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.632514    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.652311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.652290    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"]
Jul 06 19:05:11.713675 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.713613    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-j48g9"
Jul 06 19:05:11.730443 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.730417    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnmb8\" (UniqueName: \"kubernetes.io/projected/55cbd3dc-9b9d-46b3-82c6-aa3221f970c7-kube-api-access-qnmb8\") pod \"cert-manager-cainjector-559cdd8c87-rgbrm\" (UID: \"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.730546 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.730459    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55cbd3dc-9b9d-46b3-82c6-aa3221f970c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rgbrm\" (UID: \"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.831200 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.831170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnmb8\" (UniqueName: \"kubernetes.io/projected/55cbd3dc-9b9d-46b3-82c6-aa3221f970c7-kube-api-access-qnmb8\") pod \"cert-manager-cainjector-559cdd8c87-rgbrm\" (UID: \"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.831336 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.831207    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55cbd3dc-9b9d-46b3-82c6-aa3221f970c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rgbrm\" (UID: \"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.909520 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.909491    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnmb8\" (UniqueName: \"kubernetes.io/projected/55cbd3dc-9b9d-46b3-82c6-aa3221f970c7-kube-api-access-qnmb8\") pod \"cert-manager-cainjector-559cdd8c87-rgbrm\" (UID: \"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.909893 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.909874    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55cbd3dc-9b9d-46b3-82c6-aa3221f970c7-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-rgbrm\" (UID: \"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.941148 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.941126    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:05:11.949582 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:11.949563    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-j48g9"]
Jul 06 19:05:11.952354 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:05:11.952333    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1541f6e5_7700_4afb_8e59_7d03662da976.slice/crio-1021f19c813414593f9125064afd045a4c75e188b7da70d4734e4fc8aa2db535 WatchSource:0}: Error finding container 1021f19c813414593f9125064afd045a4c75e188b7da70d4734e4fc8aa2db535: Status 404 returned error can't find the container with id 1021f19c813414593f9125064afd045a4c75e188b7da70d4734e4fc8aa2db535
Jul 06 19:05:12.094874 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:12.094853    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"]
Jul 06 19:05:12.096574 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:05:12.096551    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod55cbd3dc_9b9d_46b3_82c6_aa3221f970c7.slice/crio-def00be84c0a0b95b7c6707fa3403a740087b92a2464c8ff8bd73584e9401a99 WatchSource:0}: Error finding container def00be84c0a0b95b7c6707fa3403a740087b92a2464c8ff8bd73584e9401a99: Status 404 returned error can't find the container with id def00be84c0a0b95b7c6707fa3403a740087b92a2464c8ff8bd73584e9401a99
Jul 06 19:05:12.357862 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:12.357825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm" event={"ID":"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7","Type":"ContainerStarted","Data":"def00be84c0a0b95b7c6707fa3403a740087b92a2464c8ff8bd73584e9401a99"}
Jul 06 19:05:12.358858 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:12.358830    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j48g9" event={"ID":"1541f6e5-7700-4afb-8e59-7d03662da976","Type":"ContainerStarted","Data":"1021f19c813414593f9125064afd045a4c75e188b7da70d4734e4fc8aa2db535"}
Jul 06 19:05:15.377635 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:15.377550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j48g9" event={"ID":"1541f6e5-7700-4afb-8e59-7d03662da976","Type":"ContainerStarted","Data":"85a9ece179cbc8d7bd62d2d9c3de8ee684d1198f2c3d5c0e6cb3e2a8a31f2cf6"}
Jul 06 19:05:15.378929 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:15.378908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm" event={"ID":"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7","Type":"ContainerStarted","Data":"f4b8605ecd9d84bf98a67727e4bbd9b8ad609c92dd665a21f6275419f4c5cb08"}
Jul 06 19:05:15.429125 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:15.429064    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm" podStartSLOduration=1.334405845 podStartE2EDuration="4.429043147s" totalImagesPullingTime="3.094637302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:05:15.42631791 +0000 UTC m=+375.037643449" watchObservedRunningTime="2026-07-06 19:05:15.429043147 +0000 UTC m=+375.040368687"
Jul 06 19:05:15.429285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:15.429220    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-j48g9" podStartSLOduration=1.182562872 podStartE2EDuration="4.429213392s" totalImagesPullingTime="3.24665052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:05:15.398973224 +0000 UTC m=+375.010298770" watchObservedRunningTime="2026-07-06 19:05:15.429213392 +0000 UTC m=+375.040538931"
Jul 06 19:05:22.533232 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.533192    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"]
Jul 06 19:05:22.535826 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.535804    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.538899 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.538875    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-6t7xr\""
Jul 06 19:05:22.538899 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.538883    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 06 19:05:22.539988 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.539967    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 06 19:05:22.548191 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.548171    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"]
Jul 06 19:05:22.607541 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.607515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.607739 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.607559    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4swj6\" (UniqueName: \"kubernetes.io/projected/14b2dcd5-3e11-47ba-9224-155daca64506-kube-api-access-4swj6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.607739 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.607586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.708932 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.708895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.709086 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.708947    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4swj6\" (UniqueName: \"kubernetes.io/projected/14b2dcd5-3e11-47ba-9224-155daca64506-kube-api-access-4swj6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.709086 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.708969    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.709297 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.709282    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.709335 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.709317    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.717891 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.717862    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4swj6\" (UniqueName: \"kubernetes.io/projected/14b2dcd5-3e11-47ba-9224-155daca64506-kube-api-access-4swj6\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.844623 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.844592    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:22.982762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:22.982735    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"]
Jul 06 19:05:22.984629 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:05:22.984603    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b2dcd5_3e11_47ba_9224_155daca64506.slice/crio-8507994f33906486f4fc3db33e986b746cc824b2dac666183e129facb4a90869 WatchSource:0}: Error finding container 8507994f33906486f4fc3db33e986b746cc824b2dac666183e129facb4a90869: Status 404 returned error can't find the container with id 8507994f33906486f4fc3db33e986b746cc824b2dac666183e129facb4a90869
Jul 06 19:05:23.412324 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:23.412292    2579 generic.go:356] "Generic (PLEG): container finished" podID="14b2dcd5-3e11-47ba-9224-155daca64506" containerID="ad60693b78a43782b852a9434a3dce0302ce981f7bee1e2cd2575268294deee9" exitCode=0
Jul 06 19:05:23.412515 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:23.412379    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb" event={"ID":"14b2dcd5-3e11-47ba-9224-155daca64506","Type":"ContainerDied","Data":"ad60693b78a43782b852a9434a3dce0302ce981f7bee1e2cd2575268294deee9"}
Jul 06 19:05:23.412515 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:23.412436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb" event={"ID":"14b2dcd5-3e11-47ba-9224-155daca64506","Type":"ContainerStarted","Data":"8507994f33906486f4fc3db33e986b746cc824b2dac666183e129facb4a90869"}
Jul 06 19:05:24.417746 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:24.417710    2579 generic.go:356] "Generic (PLEG): container finished" podID="14b2dcd5-3e11-47ba-9224-155daca64506" containerID="83e7d475ffedbcac33496724227a0b5d53edd3cde472895631df55c3c1a6bae4" exitCode=0
Jul 06 19:05:24.418084 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:24.417793    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb" event={"ID":"14b2dcd5-3e11-47ba-9224-155daca64506","Type":"ContainerDied","Data":"83e7d475ffedbcac33496724227a0b5d53edd3cde472895631df55c3c1a6bae4"}
Jul 06 19:05:25.423305 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:25.423271    2579 generic.go:356] "Generic (PLEG): container finished" podID="14b2dcd5-3e11-47ba-9224-155daca64506" containerID="d3bbcbc9315d7860f3310679eac5c9787c3a6b5fdd00f32240216f52c121a8c5" exitCode=0
Jul 06 19:05:25.423708 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:25.423314    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb" event={"ID":"14b2dcd5-3e11-47ba-9224-155daca64506","Type":"ContainerDied","Data":"d3bbcbc9315d7860f3310679eac5c9787c3a6b5fdd00f32240216f52c121a8c5"}
Jul 06 19:05:26.556411 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.556387    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:26.637696 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.637651    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-util\") pod \"14b2dcd5-3e11-47ba-9224-155daca64506\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") "
Jul 06 19:05:26.637835 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.637723    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-bundle\") pod \"14b2dcd5-3e11-47ba-9224-155daca64506\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") "
Jul 06 19:05:26.637835 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.637773    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4swj6\" (UniqueName: \"kubernetes.io/projected/14b2dcd5-3e11-47ba-9224-155daca64506-kube-api-access-4swj6\") pod \"14b2dcd5-3e11-47ba-9224-155daca64506\" (UID: \"14b2dcd5-3e11-47ba-9224-155daca64506\") "
Jul 06 19:05:26.638441 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.638409    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-bundle" (OuterVolumeSpecName: "bundle") pod "14b2dcd5-3e11-47ba-9224-155daca64506" (UID: "14b2dcd5-3e11-47ba-9224-155daca64506"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:05:26.639759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.639738    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b2dcd5-3e11-47ba-9224-155daca64506-kube-api-access-4swj6" (OuterVolumeSpecName: "kube-api-access-4swj6") pod "14b2dcd5-3e11-47ba-9224-155daca64506" (UID: "14b2dcd5-3e11-47ba-9224-155daca64506"). InnerVolumeSpecName "kube-api-access-4swj6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:05:26.643631 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.643600    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-util" (OuterVolumeSpecName: "util") pod "14b2dcd5-3e11-47ba-9224-155daca64506" (UID: "14b2dcd5-3e11-47ba-9224-155daca64506"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:05:26.739172 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.739099    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-util\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:05:26.739172 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.739132    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/14b2dcd5-3e11-47ba-9224-155daca64506-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:05:26.739172 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:26.739141    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4swj6\" (UniqueName: \"kubernetes.io/projected/14b2dcd5-3e11-47ba-9224-155daca64506-kube-api-access-4swj6\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:05:27.433986 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:27.433922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb" event={"ID":"14b2dcd5-3e11-47ba-9224-155daca64506","Type":"ContainerDied","Data":"8507994f33906486f4fc3db33e986b746cc824b2dac666183e129facb4a90869"}
Jul 06 19:05:27.433986 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:27.433960    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af2wxjb"
Jul 06 19:05:27.433986 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:27.433991    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8507994f33906486f4fc3db33e986b746cc824b2dac666183e129facb4a90869"
Jul 06 19:05:35.907936 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.907890    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"]
Jul 06 19:05:35.908311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.908263    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b2dcd5-3e11-47ba-9224-155daca64506" containerName="extract"
Jul 06 19:05:35.908311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.908278    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b2dcd5-3e11-47ba-9224-155daca64506" containerName="extract"
Jul 06 19:05:35.908311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.908295    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b2dcd5-3e11-47ba-9224-155daca64506" containerName="pull"
Jul 06 19:05:35.908311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.908301    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b2dcd5-3e11-47ba-9224-155daca64506" containerName="pull"
Jul 06 19:05:35.908311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.908308    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b2dcd5-3e11-47ba-9224-155daca64506" containerName="util"
Jul 06 19:05:35.908311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.908314    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b2dcd5-3e11-47ba-9224-155daca64506" containerName="util"
Jul 06 19:05:35.908492 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.908353    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b2dcd5-3e11-47ba-9224-155daca64506" containerName="extract"
Jul 06 19:05:35.910534 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.910519    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:35.913722 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.913700    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 19:05:35.918066 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.918048    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 19:05:35.918174 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.918117    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 19:05:35.918242 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.918183    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 19:05:35.918242 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.918214    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kmtgc\""
Jul 06 19:05:35.951632 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:35.951600    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"]
Jul 06 19:05:36.002636 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.002603    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/66e3b8e2-2424-4b25-91e1-d04f448437c3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.002788 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.002646    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh2w4\" (UniqueName: \"kubernetes.io/projected/66e3b8e2-2424-4b25-91e1-d04f448437c3-kube-api-access-zh2w4\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.002788 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.002686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/66e3b8e2-2424-4b25-91e1-d04f448437c3-manifests\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.002788 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.002708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e3b8e2-2424-4b25-91e1-d04f448437c3-tmp\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.002788 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.002731    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66e3b8e2-2424-4b25-91e1-d04f448437c3-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.104140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.104086    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh2w4\" (UniqueName: \"kubernetes.io/projected/66e3b8e2-2424-4b25-91e1-d04f448437c3-kube-api-access-zh2w4\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.104140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.104142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/66e3b8e2-2424-4b25-91e1-d04f448437c3-manifests\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.104393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.104265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e3b8e2-2424-4b25-91e1-d04f448437c3-tmp\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.104393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.104328    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66e3b8e2-2424-4b25-91e1-d04f448437c3-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.104500 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.104402    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/66e3b8e2-2424-4b25-91e1-d04f448437c3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.104500 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.104467    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/66e3b8e2-2424-4b25-91e1-d04f448437c3-manifests\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.106434 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.106410    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e3b8e2-2424-4b25-91e1-d04f448437c3-tmp\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.106572 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.106555    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/66e3b8e2-2424-4b25-91e1-d04f448437c3-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.106750 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.106731    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/66e3b8e2-2424-4b25-91e1-d04f448437c3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.114069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.114044    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh2w4\" (UniqueName: \"kubernetes.io/projected/66e3b8e2-2424-4b25-91e1-d04f448437c3-kube-api-access-zh2w4\") pod \"opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4\" (UID: \"66e3b8e2-2424-4b25-91e1-d04f448437c3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.220323 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.220243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:36.360477 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.360453    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"]
Jul 06 19:05:36.363109 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:05:36.363078    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e3b8e2_2424_4b25_91e1_d04f448437c3.slice/crio-d114c1d2e357bc9fcc7b56b4085d3da950bfbeebbc975a4541a6a077d760de1e WatchSource:0}: Error finding container d114c1d2e357bc9fcc7b56b4085d3da950bfbeebbc975a4541a6a077d760de1e: Status 404 returned error can't find the container with id d114c1d2e357bc9fcc7b56b4085d3da950bfbeebbc975a4541a6a077d760de1e
Jul 06 19:05:36.466472 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:36.466437    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerStarted","Data":"d114c1d2e357bc9fcc7b56b4085d3da950bfbeebbc975a4541a6a077d760de1e"}
Jul 06 19:05:39.480617 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:39.480550    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerStarted","Data":"c5dd62a3de9c2a7474ed1dadb793f991bd64c28a55027329cd67c82594d60470"}
Jul 06 19:05:40.485604 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:40.485571    2579 generic.go:356] "Generic (PLEG): container finished" podID="66e3b8e2-2424-4b25-91e1-d04f448437c3" containerID="c5dd62a3de9c2a7474ed1dadb793f991bd64c28a55027329cd67c82594d60470" exitCode=0
Jul 06 19:05:40.486011 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:40.485625    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerDied","Data":"c5dd62a3de9c2a7474ed1dadb793f991bd64c28a55027329cd67c82594d60470"}
Jul 06 19:05:41.491634 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:41.491601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerStarted","Data":"81ccaba794b609d8c1b9032f705a2f5f41f7269a57795687d21b06c44beb8b61"}
Jul 06 19:05:41.492042 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:41.491704    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:52.497492 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:52.497461    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:05:52.546986 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:52.546935    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" podStartSLOduration=13.096161195 podStartE2EDuration="17.546921725s" totalImagesPullingTime="3.371414507s" totalInitContainerRuntime="1.079346023s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:05:41.519958816 +0000 UTC m=+401.131284370" watchObservedRunningTime="2026-07-06 19:05:52.546921725 +0000 UTC m=+412.158247274"
Jul 06 19:05:56.678808 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.678770    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"]
Jul 06 19:05:56.681364 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.681341    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b88jv"]
Jul 06 19:05:56.681514 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.681452    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.683515 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.683497    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:56.731573 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.731550    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 19:05:56.731703 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.731585    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 19:05:56.732995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.732976    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 06 19:05:56.732995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.732989    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 06 19:05:56.733166 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.733025    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 19:05:56.736558 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.736540    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5trc2\""
Jul 06 19:05:56.745053 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.745024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e455cae-324d-4ec9-9aff-12704be55ebe-tls-certs\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.745053 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.745049    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e455cae-324d-4ec9-9aff-12704be55ebe-tmp\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.745198 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.745073    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpcvw\" (UniqueName: \"kubernetes.io/projected/bbf6f300-9954-49ff-8236-e44e0c7dfde2-kube-api-access-qpcvw\") pod \"odh-model-controller-7998c67dc4-b88jv\" (UID: \"bbf6f300-9954-49ff-8236-e44e0c7dfde2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:56.745198 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.745092    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf6f300-9954-49ff-8236-e44e0c7dfde2-cert\") pod \"odh-model-controller-7998c67dc4-b88jv\" (UID: \"bbf6f300-9954-49ff-8236-e44e0c7dfde2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:56.745198 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.745161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wclvn\" (UniqueName: \"kubernetes.io/projected/8e455cae-324d-4ec9-9aff-12704be55ebe-kube-api-access-wclvn\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.767878 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.767851    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"]
Jul 06 19:05:56.846261 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.846228    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e455cae-324d-4ec9-9aff-12704be55ebe-tls-certs\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.846261 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.846264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e455cae-324d-4ec9-9aff-12704be55ebe-tmp\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.846479 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.846292    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpcvw\" (UniqueName: \"kubernetes.io/projected/bbf6f300-9954-49ff-8236-e44e0c7dfde2-kube-api-access-qpcvw\") pod \"odh-model-controller-7998c67dc4-b88jv\" (UID: \"bbf6f300-9954-49ff-8236-e44e0c7dfde2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:56.846479 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.846319    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf6f300-9954-49ff-8236-e44e0c7dfde2-cert\") pod \"odh-model-controller-7998c67dc4-b88jv\" (UID: \"bbf6f300-9954-49ff-8236-e44e0c7dfde2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:56.846479 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.846360    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wclvn\" (UniqueName: \"kubernetes.io/projected/8e455cae-324d-4ec9-9aff-12704be55ebe-kube-api-access-wclvn\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.846629 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:05:56.846573    2579 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 06 19:05:56.846700 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:05:56.846633    2579 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbf6f300-9954-49ff-8236-e44e0c7dfde2-cert podName:bbf6f300-9954-49ff-8236-e44e0c7dfde2 nodeName:}" failed. No retries permitted until 2026-07-06 19:05:57.34661232 +0000 UTC m=+416.957937856 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/bbf6f300-9954-49ff-8236-e44e0c7dfde2-cert") pod "odh-model-controller-7998c67dc4-b88jv" (UID: "bbf6f300-9954-49ff-8236-e44e0c7dfde2") : secret "odh-model-controller-webhook-cert" not found
Jul 06 19:05:56.848768 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.848710    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8e455cae-324d-4ec9-9aff-12704be55ebe-tmp\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.849201 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.849179    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e455cae-324d-4ec9-9aff-12704be55ebe-tls-certs\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:56.904575 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.904542    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b88jv"]
Jul 06 19:05:56.969081 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:56.969018    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpcvw\" (UniqueName: \"kubernetes.io/projected/bbf6f300-9954-49ff-8236-e44e0c7dfde2-kube-api-access-qpcvw\") pod \"odh-model-controller-7998c67dc4-b88jv\" (UID: \"bbf6f300-9954-49ff-8236-e44e0c7dfde2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:57.059216 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.059185    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wclvn\" (UniqueName: \"kubernetes.io/projected/8e455cae-324d-4ec9-9aff-12704be55ebe-kube-api-access-wclvn\") pod \"kube-auth-proxy-7c9545dd54-rhcsh\" (UID: \"8e455cae-324d-4ec9-9aff-12704be55ebe\") " pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:57.290990 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.290957    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"
Jul 06 19:05:57.352245 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.351170    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf6f300-9954-49ff-8236-e44e0c7dfde2-cert\") pod \"odh-model-controller-7998c67dc4-b88jv\" (UID: \"bbf6f300-9954-49ff-8236-e44e0c7dfde2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:57.354360 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.354328    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bbf6f300-9954-49ff-8236-e44e0c7dfde2-cert\") pod \"odh-model-controller-7998c67dc4-b88jv\" (UID: \"bbf6f300-9954-49ff-8236-e44e0c7dfde2\") " pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:57.521745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.521716    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh"]
Jul 06 19:05:57.524016 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:05:57.523979    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e455cae_324d_4ec9_9aff_12704be55ebe.slice/crio-258989314f1e20467d6179573cc4bae351c14913ea0356c9845f358cc6fe879e WatchSource:0}: Error finding container 258989314f1e20467d6179573cc4bae351c14913ea0356c9845f358cc6fe879e: Status 404 returned error can't find the container with id 258989314f1e20467d6179573cc4bae351c14913ea0356c9845f358cc6fe879e
Jul 06 19:05:57.557302 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.557228    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh" event={"ID":"8e455cae-324d-4ec9-9aff-12704be55ebe","Type":"ContainerStarted","Data":"258989314f1e20467d6179573cc4bae351c14913ea0356c9845f358cc6fe879e"}
Jul 06 19:05:57.597507 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.597475    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:05:57.754524 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:57.754451    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-b88jv"]
Jul 06 19:05:57.756621 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:05:57.756595    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf6f300_9954_49ff_8236_e44e0c7dfde2.slice/crio-eefe96d68ba61e5c49222ec6d9f8cd5f0cea2ab4f66c5fdb90940064bd47b40b WatchSource:0}: Error finding container eefe96d68ba61e5c49222ec6d9f8cd5f0cea2ab4f66c5fdb90940064bd47b40b: Status 404 returned error can't find the container with id eefe96d68ba61e5c49222ec6d9f8cd5f0cea2ab4f66c5fdb90940064bd47b40b
Jul 06 19:05:58.573440 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:58.573357    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerStarted","Data":"eefe96d68ba61e5c49222ec6d9f8cd5f0cea2ab4f66c5fdb90940064bd47b40b"}
Jul 06 19:05:59.068901 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.065074    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bc77fb5b9-vk7zn"]
Jul 06 19:05:59.076571 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.076543    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.093127 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.092929    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 06 19:05:59.094276 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.094257    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 06 19:05:59.094938 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.094917    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 06 19:05:59.095429 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.095412    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 06 19:05:59.096095 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.096073    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 06 19:05:59.096294 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.096279    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-vzl4z\""
Jul 06 19:05:59.096648 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.096630    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 06 19:05:59.097414 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.097245    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 06 19:05:59.138098 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.138079    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 06 19:05:59.166881 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.166818    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc77fb5b9-vk7zn"]
Jul 06 19:05:59.174006 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.173975    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmrwv\" (UniqueName: \"kubernetes.io/projected/25b105dd-b380-47e1-9526-cc45814257a8-kube-api-access-fmrwv\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.174114 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.174017    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-service-ca\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.174114 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.174048    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-oauth-serving-cert\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.174114 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.174091    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b105dd-b380-47e1-9526-cc45814257a8-console-serving-cert\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.174272 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.174149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-console-config\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.174272 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.174187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b105dd-b380-47e1-9526-cc45814257a8-console-oauth-config\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.174272 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.174208    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-trusted-ca-bundle\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.275480 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.275426    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b105dd-b380-47e1-9526-cc45814257a8-console-oauth-config\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.275480 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.275477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-trusted-ca-bundle\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.275723 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.275518    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmrwv\" (UniqueName: \"kubernetes.io/projected/25b105dd-b380-47e1-9526-cc45814257a8-kube-api-access-fmrwv\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.275723 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.275550    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-service-ca\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.275723 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.275643    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-oauth-serving-cert\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.275894 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.275724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b105dd-b380-47e1-9526-cc45814257a8-console-serving-cert\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.275894 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.275788    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-console-config\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.276499 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.276474    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-oauth-serving-cert\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.276618 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.276586    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-trusted-ca-bundle\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.276993 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.276946    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-console-config\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.277249 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.277212    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/25b105dd-b380-47e1-9526-cc45814257a8-service-ca\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.279143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.279101    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/25b105dd-b380-47e1-9526-cc45814257a8-console-serving-cert\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.279143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.279113    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/25b105dd-b380-47e1-9526-cc45814257a8-console-oauth-config\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.312702 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.312672    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmrwv\" (UniqueName: \"kubernetes.io/projected/25b105dd-b380-47e1-9526-cc45814257a8-kube-api-access-fmrwv\") pod \"console-bc77fb5b9-vk7zn\" (UID: \"25b105dd-b380-47e1-9526-cc45814257a8\") " pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:05:59.388729 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:05:59.388469    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:06:00.762212 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:00.762184    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bc77fb5b9-vk7zn"]
Jul 06 19:06:00.763956 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:06:00.763929    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25b105dd_b380_47e1_9526_cc45814257a8.slice/crio-8590bf7d16d48a1c49f310e0d4d36d103140887bd815407fd7364949c4eba025 WatchSource:0}: Error finding container 8590bf7d16d48a1c49f310e0d4d36d103140887bd815407fd7364949c4eba025: Status 404 returned error can't find the container with id 8590bf7d16d48a1c49f310e0d4d36d103140887bd815407fd7364949c4eba025
Jul 06 19:06:01.588865 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:01.588829    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerStarted","Data":"79274a71e6e4dce947b6fbf87c34c849fa432dde27bab839d17f330e6e63070f"}
Jul 06 19:06:01.589068 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:01.588936    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:06:01.590425 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:01.590402    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc77fb5b9-vk7zn" event={"ID":"25b105dd-b380-47e1-9526-cc45814257a8","Type":"ContainerStarted","Data":"4afe2a06424deb95ce897c674815510406610eb5cd0201a4d33a726f5f18b335"}
Jul 06 19:06:01.590544 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:01.590440    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bc77fb5b9-vk7zn" event={"ID":"25b105dd-b380-47e1-9526-cc45814257a8","Type":"ContainerStarted","Data":"8590bf7d16d48a1c49f310e0d4d36d103140887bd815407fd7364949c4eba025"}
Jul 06 19:06:01.623624 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:01.623580    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podStartSLOduration=2.779564053 podStartE2EDuration="5.623567635s" totalImagesPullingTime="2.844003582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:01.622043015 +0000 UTC m=+421.233368555" watchObservedRunningTime="2026-07-06 19:06:01.623567635 +0000 UTC m=+421.234893173"
Jul 06 19:06:01.664464 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:01.664405    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bc77fb5b9-vk7zn" podStartSLOduration=3.664388846 podStartE2EDuration="3.664388846s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:01.663599852 +0000 UTC m=+421.274925391" watchObservedRunningTime="2026-07-06 19:06:01.664388846 +0000 UTC m=+421.275714387"
Jul 06 19:06:02.885216 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:02.885193    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 06 19:06:03.606667 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:03.606623    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh" event={"ID":"8e455cae-324d-4ec9-9aff-12704be55ebe","Type":"ContainerStarted","Data":"d000a0948878fe4343bf1f29d8242f745cbbeeccefb7e05b82f99e93f505e67d"}
Jul 06 19:06:03.685280 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:03.685230    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7c9545dd54-rhcsh" podStartSLOduration=2.32918972 podStartE2EDuration="7.68521577s" totalImagesPullingTime="5.35602605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:05:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:03.684407408 +0000 UTC m=+423.295732947" watchObservedRunningTime="2026-07-06 19:06:03.68521577 +0000 UTC m=+423.296541306"
Jul 06 19:06:04.380409 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.380376    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"]
Jul 06 19:06:04.382584 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.382569    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.387233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.387208    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 06 19:06:04.387233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.387223    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-brm4b\""
Jul 06 19:06:04.387395 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.387267    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 06 19:06:04.387395 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.387309    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 06 19:06:04.387754 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.387734    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 19:06:04.401082 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.401066    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 19:06:04.493955 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.493929    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"]
Jul 06 19:06:04.517140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.517115    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-metrics-cert\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.517237 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.517153    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-manager-config\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.517237 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.517171    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsdwh\" (UniqueName: \"kubernetes.io/projected/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-kube-api-access-tsdwh\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.517237 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.517215    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-cert\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.618210 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.618189    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-cert\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.618329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.618234    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-metrics-cert\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.618329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.618264    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-manager-config\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.618329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.618279    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsdwh\" (UniqueName: \"kubernetes.io/projected/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-kube-api-access-tsdwh\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.619010 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.618981    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-manager-config\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.620518 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.620497    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-metrics-cert\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.620608 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.620568    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-cert\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.632565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.632522    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsdwh\" (UniqueName: \"kubernetes.io/projected/1f2ac7c7-46df-4479-a0fc-bd3b25ad8787-kube-api-access-tsdwh\") pod \"lws-controller-manager-84979c6fd9-757nm\" (UID: \"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787\") " pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.691522 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.691501    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:04.838535 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:04.838511    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"]
Jul 06 19:06:04.840928 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:06:04.840905    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f2ac7c7_46df_4479_a0fc_bd3b25ad8787.slice/crio-a1f212592c0772756faae5fbd98e8006ecba222c5dc098a09ab5bbf74bc8463d WatchSource:0}: Error finding container a1f212592c0772756faae5fbd98e8006ecba222c5dc098a09ab5bbf74bc8463d: Status 404 returned error can't find the container with id a1f212592c0772756faae5fbd98e8006ecba222c5dc098a09ab5bbf74bc8463d
Jul 06 19:06:05.617750 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:05.617713    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" event={"ID":"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787","Type":"ContainerStarted","Data":"a1f212592c0772756faae5fbd98e8006ecba222c5dc098a09ab5bbf74bc8463d"}
Jul 06 19:06:06.624012 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:06.623964    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" event={"ID":"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787","Type":"ContainerStarted","Data":"7e787a4efa7a8371964b7c516e254a4bd3b6ebe0652035d80930b74e6a042f35"}
Jul 06 19:06:06.624397 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:06.624112    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:06:09.389245 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:09.389208    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:06:09.389679 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:09.389284    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:06:09.394508 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:09.394487    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:06:09.445375 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:09.445331    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" podStartSLOduration=4.038743283 podStartE2EDuration="5.445316988s" totalImagesPullingTime="1.406573705s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:06:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:06:06.785117173 +0000 UTC m=+426.396442711" watchObservedRunningTime="2026-07-06 19:06:09.445316988 +0000 UTC m=+429.056642529"
Jul 06 19:06:09.639938 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:09.639879    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bc77fb5b9-vk7zn"
Jul 06 19:06:12.603381 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:12.603356    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:06:17.630081 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:06:17.630052    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:07:33.399758 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.399722    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"]
Jul 06 19:07:33.402246 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.402228    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.404938 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.404920    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-9xf9k\""
Jul 06 19:07:33.405035 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.404969    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 19:07:33.406072 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.406054    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 19:07:33.413058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.413037    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"]
Jul 06 19:07:33.489510 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.489469    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgpxq\" (UniqueName: \"kubernetes.io/projected/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-kube-api-access-dgpxq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.489703 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.489540    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.489703 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.489586    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.590853 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.590812    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dgpxq\" (UniqueName: \"kubernetes.io/projected/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-kube-api-access-dgpxq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.591015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.590867    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.591015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.590895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.591254 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.591236    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.591291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.591264    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.602346 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.602317    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgpxq\" (UniqueName: \"kubernetes.io/projected/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-kube-api-access-dgpxq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.711493 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.711403    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:33.805823 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.805792    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"]
Jul 06 19:07:33.808314 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.808290    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.820628 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.820603    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"]
Jul 06 19:07:33.847546 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.847524    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"]
Jul 06 19:07:33.850099 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:07:33.850075    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4b70b22_d35e_4d82_aa8e_faa5bebad0c1.slice/crio-aa7721dd6018f2564a4e5dfa38c4d4199e3894eb50fae91aefcc2be924e29bcb WatchSource:0}: Error finding container aa7721dd6018f2564a4e5dfa38c4d4199e3894eb50fae91aefcc2be924e29bcb: Status 404 returned error can't find the container with id aa7721dd6018f2564a4e5dfa38c4d4199e3894eb50fae91aefcc2be924e29bcb
Jul 06 19:07:33.892650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.892623    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.892768 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.892681    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.892768 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.892719    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n47fm\" (UniqueName: \"kubernetes.io/projected/7d8b70a3-4238-4c95-85ba-173343a207fe-kube-api-access-n47fm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.987882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.987818    2579 generic.go:356] "Generic (PLEG): container finished" podID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerID="94d93d49555cc91f2db1039c4e10e0d6cdcba3592daff0b371d61375e830cf2e" exitCode=0
Jul 06 19:07:33.988031 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.987904    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9" event={"ID":"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1","Type":"ContainerDied","Data":"94d93d49555cc91f2db1039c4e10e0d6cdcba3592daff0b371d61375e830cf2e"}
Jul 06 19:07:33.988031 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.987952    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9" event={"ID":"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1","Type":"ContainerStarted","Data":"aa7721dd6018f2564a4e5dfa38c4d4199e3894eb50fae91aefcc2be924e29bcb"}
Jul 06 19:07:33.993138 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.993120    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.993241 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.993159    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n47fm\" (UniqueName: \"kubernetes.io/projected/7d8b70a3-4238-4c95-85ba-173343a207fe-kube-api-access-n47fm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.993241 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.993224    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.993535 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.993512    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:33.993620 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:33.993573    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:34.002677 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.002639    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n47fm\" (UniqueName: \"kubernetes.io/projected/7d8b70a3-4238-4c95-85ba-173343a207fe-kube-api-access-n47fm\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:34.119127 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.119096    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:34.211202 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.211169    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"]
Jul 06 19:07:34.214298 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.214273    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.230159 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.230131    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"]
Jul 06 19:07:34.247997 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.247969    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"]
Jul 06 19:07:34.250126 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:07:34.250102    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d8b70a3_4238_4c95_85ba_173343a207fe.slice/crio-845f7dae58bc26e3b1e06c4b538b7d99baec2986d350aafbc8ae47250d81db64 WatchSource:0}: Error finding container 845f7dae58bc26e3b1e06c4b538b7d99baec2986d350aafbc8ae47250d81db64: Status 404 returned error can't find the container with id 845f7dae58bc26e3b1e06c4b538b7d99baec2986d350aafbc8ae47250d81db64
Jul 06 19:07:34.295348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.295325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cdbx\" (UniqueName: \"kubernetes.io/projected/88dd27cd-dc98-4750-9d67-917967b92b26-kube-api-access-4cdbx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.295453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.295382    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.295453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.295426    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.396557 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.396521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cdbx\" (UniqueName: \"kubernetes.io/projected/88dd27cd-dc98-4750-9d67-917967b92b26-kube-api-access-4cdbx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.396757 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.396584    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.396757 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.396605    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.396931 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.396910    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.396977 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.396962    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.405790 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.405769    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cdbx\" (UniqueName: \"kubernetes.io/projected/88dd27cd-dc98-4750-9d67-917967b92b26-kube-api-access-4cdbx\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.525578 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.525498    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:34.594069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.594039    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"]
Jul 06 19:07:34.596330 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.596314    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.608218 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.608191    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"]
Jul 06 19:07:34.699781 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.699748    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.699924 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.699804    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhdrv\" (UniqueName: \"kubernetes.io/projected/273c7612-13e3-43df-9c8b-d46a8a5a7d52-kube-api-access-bhdrv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.699924 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.699881    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.763908 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.763886    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"]
Jul 06 19:07:34.800521 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.800492    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.800684 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.800558    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.800684 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.800595    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhdrv\" (UniqueName: \"kubernetes.io/projected/273c7612-13e3-43df-9c8b-d46a8a5a7d52-kube-api-access-bhdrv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.800969 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.800945    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.801030 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.800950    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.801494 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:07:34.801473    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88dd27cd_dc98_4750_9d67_917967b92b26.slice/crio-98c7e669cabbc176e40ea32d662e3b08f0158973766711fdb45fc2ffb653ec6f WatchSource:0}: Error finding container 98c7e669cabbc176e40ea32d662e3b08f0158973766711fdb45fc2ffb653ec6f: Status 404 returned error can't find the container with id 98c7e669cabbc176e40ea32d662e3b08f0158973766711fdb45fc2ffb653ec6f
Jul 06 19:07:34.810310 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.810289    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhdrv\" (UniqueName: \"kubernetes.io/projected/273c7612-13e3-43df-9c8b-d46a8a5a7d52-kube-api-access-bhdrv\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.919001 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.918968    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:34.994966 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.994937    2579 generic.go:356] "Generic (PLEG): container finished" podID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerID="b7e305029f22c33439104b3920118f2e86f3c7030e57bf5cb89233b31a461898" exitCode=0
Jul 06 19:07:34.995135 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.995029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb" event={"ID":"7d8b70a3-4238-4c95-85ba-173343a207fe","Type":"ContainerDied","Data":"b7e305029f22c33439104b3920118f2e86f3c7030e57bf5cb89233b31a461898"}
Jul 06 19:07:34.995135 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.995087    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb" event={"ID":"7d8b70a3-4238-4c95-85ba-173343a207fe","Type":"ContainerStarted","Data":"845f7dae58bc26e3b1e06c4b538b7d99baec2986d350aafbc8ae47250d81db64"}
Jul 06 19:07:34.997031 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.997007    2579 generic.go:356] "Generic (PLEG): container finished" podID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerID="1c2f8f46eb3eb8502e3df069d956252037fec64b123d3c95353567e3e2e02ed7" exitCode=0
Jul 06 19:07:34.997164 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:34.997040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9" event={"ID":"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1","Type":"ContainerDied","Data":"1c2f8f46eb3eb8502e3df069d956252037fec64b123d3c95353567e3e2e02ed7"}
Jul 06 19:07:35.006759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:35.000058    2579 generic.go:356] "Generic (PLEG): container finished" podID="88dd27cd-dc98-4750-9d67-917967b92b26" containerID="46a31233f0ed0078158283fb8e18afa915edcc58cb6d94dc60191d0032132e86" exitCode=0
Jul 06 19:07:35.006759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:35.000278    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff" event={"ID":"88dd27cd-dc98-4750-9d67-917967b92b26","Type":"ContainerDied","Data":"46a31233f0ed0078158283fb8e18afa915edcc58cb6d94dc60191d0032132e86"}
Jul 06 19:07:35.006759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:35.000361    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff" event={"ID":"88dd27cd-dc98-4750-9d67-917967b92b26","Type":"ContainerStarted","Data":"98c7e669cabbc176e40ea32d662e3b08f0158973766711fdb45fc2ffb653ec6f"}
Jul 06 19:07:35.067849 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:35.067810    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"]
Jul 06 19:07:35.071928 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:07:35.071904    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273c7612_13e3_43df_9c8b_d46a8a5a7d52.slice/crio-cc63cae9eeda1dbb958affe37ceab1a1fcbd951a60df5cb354b24a338830e83d WatchSource:0}: Error finding container cc63cae9eeda1dbb958affe37ceab1a1fcbd951a60df5cb354b24a338830e83d: Status 404 returned error can't find the container with id cc63cae9eeda1dbb958affe37ceab1a1fcbd951a60df5cb354b24a338830e83d
Jul 06 19:07:36.006590 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.006496    2579 generic.go:356] "Generic (PLEG): container finished" podID="88dd27cd-dc98-4750-9d67-917967b92b26" containerID="ff0d3d57585d0c6e3713ff229159deec8efa7501b277bbfae6ac008c842b452b" exitCode=0
Jul 06 19:07:36.006590 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.006574    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff" event={"ID":"88dd27cd-dc98-4750-9d67-917967b92b26","Type":"ContainerDied","Data":"ff0d3d57585d0c6e3713ff229159deec8efa7501b277bbfae6ac008c842b452b"}
Jul 06 19:07:36.008347 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.008326    2579 generic.go:356] "Generic (PLEG): container finished" podID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerID="70a32e26e1879833f4658274a8b12948b2e93012bafdace1a757a8d7f7025b8e" exitCode=0
Jul 06 19:07:36.008434 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.008349    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb" event={"ID":"7d8b70a3-4238-4c95-85ba-173343a207fe","Type":"ContainerDied","Data":"70a32e26e1879833f4658274a8b12948b2e93012bafdace1a757a8d7f7025b8e"}
Jul 06 19:07:36.010545 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.010527    2579 generic.go:356] "Generic (PLEG): container finished" podID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerID="38000497666b6e1009bf5fd6f8b7d48317d52ec436b24f71506405b4ff7c7f81" exitCode=0
Jul 06 19:07:36.010648 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.010607    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9" event={"ID":"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1","Type":"ContainerDied","Data":"38000497666b6e1009bf5fd6f8b7d48317d52ec436b24f71506405b4ff7c7f81"}
Jul 06 19:07:36.012084 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.012067    2579 generic.go:356] "Generic (PLEG): container finished" podID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerID="ac5994830c0f89446d2f0087d8742069b45a108fe60982f6c8947f0132997310" exitCode=0
Jul 06 19:07:36.012200 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.012092    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj" event={"ID":"273c7612-13e3-43df-9c8b-d46a8a5a7d52","Type":"ContainerDied","Data":"ac5994830c0f89446d2f0087d8742069b45a108fe60982f6c8947f0132997310"}
Jul 06 19:07:36.012200 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:36.012114    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj" event={"ID":"273c7612-13e3-43df-9c8b-d46a8a5a7d52","Type":"ContainerStarted","Data":"cc63cae9eeda1dbb958affe37ceab1a1fcbd951a60df5cb354b24a338830e83d"}
Jul 06 19:07:37.018464 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.018439    2579 generic.go:356] "Generic (PLEG): container finished" podID="88dd27cd-dc98-4750-9d67-917967b92b26" containerID="9cbb8284e77d02cd17178be6b8080332f568799f349d2e7f712aee782cb47453" exitCode=0
Jul 06 19:07:37.018807 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.018519    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff" event={"ID":"88dd27cd-dc98-4750-9d67-917967b92b26","Type":"ContainerDied","Data":"9cbb8284e77d02cd17178be6b8080332f568799f349d2e7f712aee782cb47453"}
Jul 06 19:07:37.020553 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.020530    2579 generic.go:356] "Generic (PLEG): container finished" podID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerID="c5a668619d66e2bf4a762dd123622fb27f7a283aba149f0e91752fbe691f3560" exitCode=0
Jul 06 19:07:37.020689 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.020617    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb" event={"ID":"7d8b70a3-4238-4c95-85ba-173343a207fe","Type":"ContainerDied","Data":"c5a668619d66e2bf4a762dd123622fb27f7a283aba149f0e91752fbe691f3560"}
Jul 06 19:07:37.150491 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.150467    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:37.324077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.324048    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-bundle\") pod \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") "
Jul 06 19:07:37.324282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.324116    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dgpxq\" (UniqueName: \"kubernetes.io/projected/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-kube-api-access-dgpxq\") pod \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") "
Jul 06 19:07:37.324282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.324143    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-util\") pod \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\" (UID: \"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1\") "
Jul 06 19:07:37.324616 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.324581    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-bundle" (OuterVolumeSpecName: "bundle") pod "c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" (UID: "c4b70b22-d35e-4d82-aa8e-faa5bebad0c1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:37.326255 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.326232    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-kube-api-access-dgpxq" (OuterVolumeSpecName: "kube-api-access-dgpxq") pod "c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" (UID: "c4b70b22-d35e-4d82-aa8e-faa5bebad0c1"). InnerVolumeSpecName "kube-api-access-dgpxq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:07:37.329319 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.329286    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-util" (OuterVolumeSpecName: "util") pod "c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" (UID: "c4b70b22-d35e-4d82-aa8e-faa5bebad0c1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:37.425746 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.425713    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:37.425746 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.425741    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dgpxq\" (UniqueName: \"kubernetes.io/projected/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-kube-api-access-dgpxq\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:37.425746 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:37.425751    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c4b70b22-d35e-4d82-aa8e-faa5bebad0c1-util\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:38.028754 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.028668    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9" event={"ID":"c4b70b22-d35e-4d82-aa8e-faa5bebad0c1","Type":"ContainerDied","Data":"aa7721dd6018f2564a4e5dfa38c4d4199e3894eb50fae91aefcc2be924e29bcb"}
Jul 06 19:07:38.028754 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.028715    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa7721dd6018f2564a4e5dfa38c4d4199e3894eb50fae91aefcc2be924e29bcb"
Jul 06 19:07:38.028754 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.028682    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9"
Jul 06 19:07:38.030443 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.030420    2579 generic.go:356] "Generic (PLEG): container finished" podID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerID="ff1473e1797cf32bee7d81016f244fd2d916471ec5f9b7d48abcdba20861f5ac" exitCode=0
Jul 06 19:07:38.030553 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.030487    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj" event={"ID":"273c7612-13e3-43df-9c8b-d46a8a5a7d52","Type":"ContainerDied","Data":"ff1473e1797cf32bee7d81016f244fd2d916471ec5f9b7d48abcdba20861f5ac"}
Jul 06 19:07:38.165169 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.165143    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:38.191062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.191038    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:38.332583 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.332549    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-bundle\") pod \"7d8b70a3-4238-4c95-85ba-173343a207fe\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") "
Jul 06 19:07:38.332583 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.332587    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-bundle\") pod \"88dd27cd-dc98-4750-9d67-917967b92b26\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") "
Jul 06 19:07:38.332846 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.332636    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-util\") pod \"88dd27cd-dc98-4750-9d67-917967b92b26\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") "
Jul 06 19:07:38.332846 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.332679    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cdbx\" (UniqueName: \"kubernetes.io/projected/88dd27cd-dc98-4750-9d67-917967b92b26-kube-api-access-4cdbx\") pod \"88dd27cd-dc98-4750-9d67-917967b92b26\" (UID: \"88dd27cd-dc98-4750-9d67-917967b92b26\") "
Jul 06 19:07:38.332846 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.332717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-util\") pod \"7d8b70a3-4238-4c95-85ba-173343a207fe\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") "
Jul 06 19:07:38.332846 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.332755    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n47fm\" (UniqueName: \"kubernetes.io/projected/7d8b70a3-4238-4c95-85ba-173343a207fe-kube-api-access-n47fm\") pod \"7d8b70a3-4238-4c95-85ba-173343a207fe\" (UID: \"7d8b70a3-4238-4c95-85ba-173343a207fe\") "
Jul 06 19:07:38.333204 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.333170    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-bundle" (OuterVolumeSpecName: "bundle") pod "7d8b70a3-4238-4c95-85ba-173343a207fe" (UID: "7d8b70a3-4238-4c95-85ba-173343a207fe"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:38.333322 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.333175    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-bundle" (OuterVolumeSpecName: "bundle") pod "88dd27cd-dc98-4750-9d67-917967b92b26" (UID: "88dd27cd-dc98-4750-9d67-917967b92b26"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:38.335362 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.335336    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88dd27cd-dc98-4750-9d67-917967b92b26-kube-api-access-4cdbx" (OuterVolumeSpecName: "kube-api-access-4cdbx") pod "88dd27cd-dc98-4750-9d67-917967b92b26" (UID: "88dd27cd-dc98-4750-9d67-917967b92b26"). InnerVolumeSpecName "kube-api-access-4cdbx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:07:38.335431 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.335373    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d8b70a3-4238-4c95-85ba-173343a207fe-kube-api-access-n47fm" (OuterVolumeSpecName: "kube-api-access-n47fm") pod "7d8b70a3-4238-4c95-85ba-173343a207fe" (UID: "7d8b70a3-4238-4c95-85ba-173343a207fe"). InnerVolumeSpecName "kube-api-access-n47fm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:07:38.338352 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.338329    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-util" (OuterVolumeSpecName: "util") pod "88dd27cd-dc98-4750-9d67-917967b92b26" (UID: "88dd27cd-dc98-4750-9d67-917967b92b26"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:38.338623 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.338603    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-util" (OuterVolumeSpecName: "util") pod "7d8b70a3-4238-4c95-85ba-173343a207fe" (UID: "7d8b70a3-4238-4c95-85ba-173343a207fe"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:38.433878 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.433848    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-util\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:38.433878 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.433877    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n47fm\" (UniqueName: \"kubernetes.io/projected/7d8b70a3-4238-4c95-85ba-173343a207fe-kube-api-access-n47fm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:38.434064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.433887    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7d8b70a3-4238-4c95-85ba-173343a207fe-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:38.434064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.433897    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:38.434064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.433904    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/88dd27cd-dc98-4750-9d67-917967b92b26-util\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:38.434064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:38.433912    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cdbx\" (UniqueName: \"kubernetes.io/projected/88dd27cd-dc98-4750-9d67-917967b92b26-kube-api-access-4cdbx\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:39.037619 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.037587    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb"
Jul 06 19:07:39.037619 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.037612    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb" event={"ID":"7d8b70a3-4238-4c95-85ba-173343a207fe","Type":"ContainerDied","Data":"845f7dae58bc26e3b1e06c4b538b7d99baec2986d350aafbc8ae47250d81db64"}
Jul 06 19:07:39.038144 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.037682    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="845f7dae58bc26e3b1e06c4b538b7d99baec2986d350aafbc8ae47250d81db64"
Jul 06 19:07:39.040072 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.040045    2579 generic.go:356] "Generic (PLEG): container finished" podID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerID="66110b08de9e519e66a43eab1c7984d135ad6f705c688e9e42c20a7477b24e71" exitCode=0
Jul 06 19:07:39.040197 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.040112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj" event={"ID":"273c7612-13e3-43df-9c8b-d46a8a5a7d52","Type":"ContainerDied","Data":"66110b08de9e519e66a43eab1c7984d135ad6f705c688e9e42c20a7477b24e71"}
Jul 06 19:07:39.041795 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.041777    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff" event={"ID":"88dd27cd-dc98-4750-9d67-917967b92b26","Type":"ContainerDied","Data":"98c7e669cabbc176e40ea32d662e3b08f0158973766711fdb45fc2ffb653ec6f"}
Jul 06 19:07:39.041893 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.041804    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff"
Jul 06 19:07:39.041893 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:39.041808    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98c7e669cabbc176e40ea32d662e3b08f0158973766711fdb45fc2ffb653ec6f"
Jul 06 19:07:40.167902 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.167879    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:40.347445 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.347352    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bhdrv\" (UniqueName: \"kubernetes.io/projected/273c7612-13e3-43df-9c8b-d46a8a5a7d52-kube-api-access-bhdrv\") pod \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") "
Jul 06 19:07:40.347445 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.347396    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-bundle\") pod \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") "
Jul 06 19:07:40.347445 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.347420    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-util\") pod \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\" (UID: \"273c7612-13e3-43df-9c8b-d46a8a5a7d52\") "
Jul 06 19:07:40.348139 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.348105    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-bundle" (OuterVolumeSpecName: "bundle") pod "273c7612-13e3-43df-9c8b-d46a8a5a7d52" (UID: "273c7612-13e3-43df-9c8b-d46a8a5a7d52"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:40.349408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.349382    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/273c7612-13e3-43df-9c8b-d46a8a5a7d52-kube-api-access-bhdrv" (OuterVolumeSpecName: "kube-api-access-bhdrv") pod "273c7612-13e3-43df-9c8b-d46a8a5a7d52" (UID: "273c7612-13e3-43df-9c8b-d46a8a5a7d52"). InnerVolumeSpecName "kube-api-access-bhdrv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:07:40.352630 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.352608    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-util" (OuterVolumeSpecName: "util") pod "273c7612-13e3-43df-9c8b-d46a8a5a7d52" (UID: "273c7612-13e3-43df-9c8b-d46a8a5a7d52"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:07:40.448478 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.448442    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bhdrv\" (UniqueName: \"kubernetes.io/projected/273c7612-13e3-43df-9c8b-d46a8a5a7d52-kube-api-access-bhdrv\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:40.448478 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.448471    2579 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-bundle\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:40.448478 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:40.448480    2579 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/273c7612-13e3-43df-9c8b-d46a8a5a7d52-util\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:07:41.052857 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:41.052815    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj" event={"ID":"273c7612-13e3-43df-9c8b-d46a8a5a7d52","Type":"ContainerDied","Data":"cc63cae9eeda1dbb958affe37ceab1a1fcbd951a60df5cb354b24a338830e83d"}
Jul 06 19:07:41.052857 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:41.052846    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj"
Jul 06 19:07:41.053104 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:07:41.052891    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cc63cae9eeda1dbb958affe37ceab1a1fcbd951a60df5cb354b24a338830e83d"
Jul 06 19:09:12.463888 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.463849    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f2ac7c7-46df-4479-a0fc-bd3b25ad8787" containerID="7e787a4efa7a8371964b7c516e254a4bd3b6ebe0652035d80930b74e6a042f35" exitCode=1
Jul 06 19:09:12.464502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.463920    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" event={"ID":"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787","Type":"ContainerDied","Data":"7e787a4efa7a8371964b7c516e254a4bd3b6ebe0652035d80930b74e6a042f35"}
Jul 06 19:09:12.464502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.464371    2579 scope.go:117] "RemoveContainer" containerID="7e787a4efa7a8371964b7c516e254a4bd3b6ebe0652035d80930b74e6a042f35"
Jul 06 19:09:12.466132 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.466112    2579 generic.go:356] "Generic (PLEG): container finished" podID="66e3b8e2-2424-4b25-91e1-d04f448437c3" containerID="81ccaba794b609d8c1b9032f705a2f5f41f7269a57795687d21b06c44beb8b61" exitCode=1
Jul 06 19:09:12.466219 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.466190    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerDied","Data":"81ccaba794b609d8c1b9032f705a2f5f41f7269a57795687d21b06c44beb8b61"}
Jul 06 19:09:12.466544 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.466520    2579 scope.go:117] "RemoveContainer" containerID="81ccaba794b609d8c1b9032f705a2f5f41f7269a57795687d21b06c44beb8b61"
Jul 06 19:09:12.467770 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.467753    2579 generic.go:356] "Generic (PLEG): container finished" podID="bbf6f300-9954-49ff-8236-e44e0c7dfde2" containerID="79274a71e6e4dce947b6fbf87c34c849fa432dde27bab839d17f330e6e63070f" exitCode=1
Jul 06 19:09:12.467857 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.467784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerDied","Data":"79274a71e6e4dce947b6fbf87c34c849fa432dde27bab839d17f330e6e63070f"}
Jul 06 19:09:12.468040 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.468028    2579 scope.go:117] "RemoveContainer" containerID="79274a71e6e4dce947b6fbf87c34c849fa432dde27bab839d17f330e6e63070f"
Jul 06 19:09:12.496034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.495999    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:09:12.601987 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:12.601953    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:09:13.474019 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:13.473984    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerStarted","Data":"a9efa77ed01fec3a0ba26d2e480ca4a803d5894d41ed7e65b4828bb962de1ca8"}
Jul 06 19:09:13.474417 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:13.474086    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:09:13.475761 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:13.475740    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" event={"ID":"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787","Type":"ContainerStarted","Data":"166a5ab491a5cf8661e3409b4a5e0ab9d99f3dc2899afcb1cea677142b179587"}
Jul 06 19:09:13.475930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:13.475909    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:09:13.477783 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:13.477761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerStarted","Data":"85f30d8c1ede7dcc6c0eb0ea8182a9ab026e9634266db83576ecf73be014f017"}
Jul 06 19:09:13.477897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:13.477885    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:09:13.986446 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:09:13.986406    2579 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-142-165.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:09:14.048135 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:09:14.048106    2579 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-142-165.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again"
Jul 06 19:09:24.484085 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:24.484014    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:09:24.486351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:24.484112    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:09:24.486351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:24.484272    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:09:34.267717 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:09:34.267673    2579 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-142-165.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 06 19:09:35.412473 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:09:35.412438    2579 controller.go:195] "Failed to update lease" err="Operation cannot be fulfilled on leases.coordination.k8s.io \"ip-10-0-142-165.ec2.internal\": the object has been modified; please apply your changes to the latest version and try again"
Jul 06 19:09:54.026610 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026576    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mw2l"]
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026883    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88dd27cd-dc98-4750-9d67-917967b92b26" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026896    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="88dd27cd-dc98-4750-9d67-917967b92b26" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026911    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88dd27cd-dc98-4750-9d67-917967b92b26" containerName="extract"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026917    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="88dd27cd-dc98-4750-9d67-917967b92b26" containerName="extract"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026923    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026928    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026938    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerName="extract"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026946    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerName="extract"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026955    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026960    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026967    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026971    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026978    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88dd27cd-dc98-4750-9d67-917967b92b26" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026983    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="88dd27cd-dc98-4750-9d67-917967b92b26" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026992    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.026997    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027004    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027009    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerName="pull"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027018    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027026    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerName="util"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027035    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerName="extract"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027040    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerName="extract"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027046    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerName="extract"
Jul 06 19:09:54.027052 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027050    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerName="extract"
Jul 06 19:09:54.027752 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027099    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d8b70a3-4238-4c95-85ba-173343a207fe" containerName="extract"
Jul 06 19:09:54.027752 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027109    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4b70b22-d35e-4d82-aa8e-faa5bebad0c1" containerName="extract"
Jul 06 19:09:54.027752 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027115    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="88dd27cd-dc98-4750-9d67-917967b92b26" containerName="extract"
Jul 06 19:09:54.027752 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.027121    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="273c7612-13e3-43df-9c8b-d46a8a5a7d52" containerName="extract"
Jul 06 19:09:54.031532 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.031513    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mw2l"
Jul 06 19:09:54.039312 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.039292    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 19:09:54.039883 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.039867    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 19:09:54.052795 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.052778    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-7c65r\""
Jul 06 19:09:54.053698 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.053676    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mw2l"]
Jul 06 19:09:54.128731 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.128701    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dml4j\" (UniqueName: \"kubernetes.io/projected/96b52bc7-a7a4-45c1-8d93-ec256f4b095b-kube-api-access-dml4j\") pod \"authorino-75d556bd77-7mw2l\" (UID: \"96b52bc7-a7a4-45c1-8d93-ec256f4b095b\") " pod="kuadrant-system/authorino-75d556bd77-7mw2l"
Jul 06 19:09:54.229314 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.229273    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dml4j\" (UniqueName: \"kubernetes.io/projected/96b52bc7-a7a4-45c1-8d93-ec256f4b095b-kube-api-access-dml4j\") pod \"authorino-75d556bd77-7mw2l\" (UID: \"96b52bc7-a7a4-45c1-8d93-ec256f4b095b\") " pod="kuadrant-system/authorino-75d556bd77-7mw2l"
Jul 06 19:09:54.239589 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.239555    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dml4j\" (UniqueName: \"kubernetes.io/projected/96b52bc7-a7a4-45c1-8d93-ec256f4b095b-kube-api-access-dml4j\") pod \"authorino-75d556bd77-7mw2l\" (UID: \"96b52bc7-a7a4-45c1-8d93-ec256f4b095b\") " pod="kuadrant-system/authorino-75d556bd77-7mw2l"
Jul 06 19:09:54.340407 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.340379    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mw2l"
Jul 06 19:09:54.487291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.487263    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mw2l"]
Jul 06 19:09:54.488594 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:09:54.488568    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod96b52bc7_a7a4_45c1_8d93_ec256f4b095b.slice/crio-b49eff67db52c9561a89259d5b1074a8019ba62872a6601d599b06999b61436f WatchSource:0}: Error finding container b49eff67db52c9561a89259d5b1074a8019ba62872a6601d599b06999b61436f: Status 404 returned error can't find the container with id b49eff67db52c9561a89259d5b1074a8019ba62872a6601d599b06999b61436f
Jul 06 19:09:54.489930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.489913    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:09:54.668667 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:09:54.668569    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mw2l" event={"ID":"96b52bc7-a7a4-45c1-8d93-ec256f4b095b","Type":"ContainerStarted","Data":"b49eff67db52c9561a89259d5b1074a8019ba62872a6601d599b06999b61436f"}
Jul 06 19:10:00.702615 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:00.702573    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mw2l" event={"ID":"96b52bc7-a7a4-45c1-8d93-ec256f4b095b","Type":"ContainerStarted","Data":"f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297"}
Jul 06 19:10:00.732185 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:00.732133    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-7mw2l" podStartSLOduration=2.13241058 podStartE2EDuration="7.73211757s" totalImagesPullingTime="5.59970699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:10:00.731642831 +0000 UTC m=+660.342968371" watchObservedRunningTime="2026-07-06 19:10:00.73211757 +0000 UTC m=+660.343443157"
Jul 06 19:10:43.909637 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.909608    2579 generic.go:356] "Generic (PLEG): container finished" podID="bbf6f300-9954-49ff-8236-e44e0c7dfde2" containerID="a9efa77ed01fec3a0ba26d2e480ca4a803d5894d41ed7e65b4828bb962de1ca8" exitCode=1
Jul 06 19:10:43.910073 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.909681    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerDied","Data":"a9efa77ed01fec3a0ba26d2e480ca4a803d5894d41ed7e65b4828bb962de1ca8"}
Jul 06 19:10:43.910073 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.909736    2579 scope.go:117] "RemoveContainer" containerID="79274a71e6e4dce947b6fbf87c34c849fa432dde27bab839d17f330e6e63070f"
Jul 06 19:10:43.910073 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.910051    2579 scope.go:117] "RemoveContainer" containerID="a9efa77ed01fec3a0ba26d2e480ca4a803d5894d41ed7e65b4828bb962de1ca8"
Jul 06 19:10:43.910270 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:10:43.910249    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:10:43.911952 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.911933    2579 generic.go:356] "Generic (PLEG): container finished" podID="66e3b8e2-2424-4b25-91e1-d04f448437c3" containerID="85f30d8c1ede7dcc6c0eb0ea8182a9ab026e9634266db83576ecf73be014f017" exitCode=1
Jul 06 19:10:43.912065 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.912002    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerDied","Data":"85f30d8c1ede7dcc6c0eb0ea8182a9ab026e9634266db83576ecf73be014f017"}
Jul 06 19:10:43.912330 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.912316    2579 scope.go:117] "RemoveContainer" containerID="85f30d8c1ede7dcc6c0eb0ea8182a9ab026e9634266db83576ecf73be014f017"
Jul 06 19:10:43.912484 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:10:43.912463    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_opendatahub(66e3b8e2-2424-4b25-91e1-d04f448437c3)\"" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" podUID="66e3b8e2-2424-4b25-91e1-d04f448437c3"
Jul 06 19:10:43.919847 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:43.919828    2579 scope.go:117] "RemoveContainer" containerID="81ccaba794b609d8c1b9032f705a2f5f41f7269a57795687d21b06c44beb8b61"
Jul 06 19:10:44.483178 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:44.483139    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:10:44.483342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:44.483201    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:10:44.918705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:44.918679    2579 scope.go:117] "RemoveContainer" containerID="85f30d8c1ede7dcc6c0eb0ea8182a9ab026e9634266db83576ecf73be014f017"
Jul 06 19:10:44.919137 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:10:44.918870    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_opendatahub(66e3b8e2-2424-4b25-91e1-d04f448437c3)\"" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" podUID="66e3b8e2-2424-4b25-91e1-d04f448437c3"
Jul 06 19:10:44.920137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:44.920118    2579 scope.go:117] "RemoveContainer" containerID="a9efa77ed01fec3a0ba26d2e480ca4a803d5894d41ed7e65b4828bb962de1ca8"
Jul 06 19:10:44.920293 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:10:44.920276    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:10:49.656942 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:10:49.656903    2579 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-142-165.ec2.internal?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:10:50.488965 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:50.488916    2579 prober.go:120] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-165.ec2.internal" podUID="7a9b387f1ea494ba8e3440d0784de7c5" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded"
Jul 06 19:10:55.922357 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:55.922278    2579 scope.go:117] "RemoveContainer" containerID="85f30d8c1ede7dcc6c0eb0ea8182a9ab026e9634266db83576ecf73be014f017"
Jul 06 19:10:56.220849 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:56.220790    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:10:56.976552 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:56.976517    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerStarted","Data":"93a318508caffc925fbad2623914459a46bc472eab6972cbf999ae38e2427b15"}
Jul 06 19:10:56.976948 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:56.976694    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:10:57.597943 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:57.597912    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:10:57.598301 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:57.598286    2579 scope.go:117] "RemoveContainer" containerID="a9efa77ed01fec3a0ba26d2e480ca4a803d5894d41ed7e65b4828bb962de1ca8"
Jul 06 19:10:58.988803 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:58.988766    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerStarted","Data":"80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b"}
Jul 06 19:10:58.989202 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:10:58.988981    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:10:59.658104 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:10:59.658062    2579 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-142-165.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 06 19:11:07.026717 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:07.026683    2579 generic.go:356] "Generic (PLEG): container finished" podID="66e3b8e2-2424-4b25-91e1-d04f448437c3" containerID="93a318508caffc925fbad2623914459a46bc472eab6972cbf999ae38e2427b15" exitCode=1
Jul 06 19:11:07.027143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:07.026751    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerDied","Data":"93a318508caffc925fbad2623914459a46bc472eab6972cbf999ae38e2427b15"}
Jul 06 19:11:07.027143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:07.026809    2579 scope.go:117] "RemoveContainer" containerID="85f30d8c1ede7dcc6c0eb0ea8182a9ab026e9634266db83576ecf73be014f017"
Jul 06 19:11:07.027245 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:07.027170    2579 scope.go:117] "RemoveContainer" containerID="93a318508caffc925fbad2623914459a46bc472eab6972cbf999ae38e2427b15"
Jul 06 19:11:07.027358 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:07.027334    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_opendatahub(66e3b8e2-2424-4b25-91e1-d04f448437c3)\"" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" podUID="66e3b8e2-2424-4b25-91e1-d04f448437c3"
Jul 06 19:11:09.039519 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:09.039485    2579 generic.go:356] "Generic (PLEG): container finished" podID="bbf6f300-9954-49ff-8236-e44e0c7dfde2" containerID="80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b" exitCode=1
Jul 06 19:11:09.039915 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:09.039559    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerDied","Data":"80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b"}
Jul 06 19:11:09.039915 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:09.039615    2579 scope.go:117] "RemoveContainer" containerID="a9efa77ed01fec3a0ba26d2e480ca4a803d5894d41ed7e65b4828bb962de1ca8"
Jul 06 19:11:09.040002 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:09.039915    2579 scope.go:117] "RemoveContainer" containerID="80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b"
Jul 06 19:11:09.040134 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:09.040116    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:11:09.658716 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:09.658683    2579 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-142-165.ec2.internal?timeout=10s\": context deadline exceeded"
Jul 06 19:11:16.220457 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:16.220417    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:11:16.220879 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:16.220849    2579 scope.go:117] "RemoveContainer" containerID="93a318508caffc925fbad2623914459a46bc472eab6972cbf999ae38e2427b15"
Jul 06 19:11:16.221044 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:16.221024    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_opendatahub(66e3b8e2-2424-4b25-91e1-d04f448437c3)\"" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" podUID="66e3b8e2-2424-4b25-91e1-d04f448437c3"
Jul 06 19:11:17.598328 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:17.598293    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:11:17.598724 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:17.598610    2579 scope.go:117] "RemoveContainer" containerID="80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b"
Jul 06 19:11:17.598830 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:17.598810    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 20s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:11:19.659406 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:19.659367    2579 controller.go:195] "Failed to update lease" err="Put \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/ip-10-0-142-165.ec2.internal?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:11:26.922361 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:26.922331    2579 scope.go:117] "RemoveContainer" containerID="93a318508caffc925fbad2623914459a46bc472eab6972cbf999ae38e2427b15"
Jul 06 19:11:27.882267 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:27.882216    2579 kubelet_node_status.go:536] "Error updating node status, will retry" err="failed to patch status \"{\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"type\\\":\\\"DiskPressure\\\"},{\\\"type\\\":\\\"PIDPressure\\\"},{\\\"type\\\":\\\"Ready\\\"}],\\\"conditions\\\":[{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:17Z\\\",\\\"type\\\":\\\"MemoryPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:17Z\\\",\\\"type\\\":\\\"DiskPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:17Z\\\",\\\"type\\\":\\\"PIDPressure\\\"},{\\\"lastHeartbeatTime\\\":\\\"2026-07-06T19:11:17Z\\\",\\\"type\\\":\\\"Ready\\\"}],\\\"runtimeHandlers\\\":[{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"runc\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"crun\\\"},{\\\"features\\\":{\\\"recursiveReadOnlyMounts\\\":true,\\\"userNamespaces\\\":true},\\\"name\\\":\\\"\\\"}]}}\" for node \"ip-10-0-142-165.ec2.internal\": Patch \"https://aa02cc370fcfb46c9bb69b2b56b189b2-b1b7f0f080e2fe4d.elb.us-east-1.amazonaws.com:6443/api/v1/nodes/ip-10-0-142-165.ec2.internal/status?timeout=10s\": context deadline exceeded"
Jul 06 19:11:28.130923 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:28.130883    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4" event={"ID":"66e3b8e2-2424-4b25-91e1-d04f448437c3","Type":"ContainerStarted","Data":"cad7aa8a901ca8c9b096561ff4d40cef73756cb4314b11d745e0cfeebca722f4"}
Jul 06 19:11:28.131385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:28.131229    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:11:28.922306 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:28.922271    2579 scope.go:117] "RemoveContainer" containerID="80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b"
Jul 06 19:11:30.141247 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:30.141208    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerStarted","Data":"5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"}
Jul 06 19:11:30.141730 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:30.141413    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:11:38.178474 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:38.178436    2579 generic.go:356] "Generic (PLEG): container finished" podID="55cbd3dc-9b9d-46b3-82c6-aa3221f970c7" containerID="f4b8605ecd9d84bf98a67727e4bbd9b8ad609c92dd665a21f6275419f4c5cb08" exitCode=1
Jul 06 19:11:38.178872 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:38.178507    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm" event={"ID":"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7","Type":"ContainerDied","Data":"f4b8605ecd9d84bf98a67727e4bbd9b8ad609c92dd665a21f6275419f4c5cb08"}
Jul 06 19:11:38.178918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:38.178881    2579 scope.go:117] "RemoveContainer" containerID="f4b8605ecd9d84bf98a67727e4bbd9b8ad609c92dd665a21f6275419f4c5cb08"
Jul 06 19:11:38.180308 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:38.180282    2579 generic.go:356] "Generic (PLEG): container finished" podID="1541f6e5-7700-4afb-8e59-7d03662da976" containerID="85a9ece179cbc8d7bd62d2d9c3de8ee684d1198f2c3d5c0e6cb3e2a8a31f2cf6" exitCode=1
Jul 06 19:11:38.180409 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:38.180366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j48g9" event={"ID":"1541f6e5-7700-4afb-8e59-7d03662da976","Type":"ContainerDied","Data":"85a9ece179cbc8d7bd62d2d9c3de8ee684d1198f2c3d5c0e6cb3e2a8a31f2cf6"}
Jul 06 19:11:38.180637 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:38.180621    2579 scope.go:117] "RemoveContainer" containerID="85a9ece179cbc8d7bd62d2d9c3de8ee684d1198f2c3d5c0e6cb3e2a8a31f2cf6"
Jul 06 19:11:39.137805 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:39.137776    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4"
Jul 06 19:11:39.189238 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:39.189200    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm" event={"ID":"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7","Type":"ContainerStarted","Data":"9cdefd824a9f5151278746f53a601bb4c9595baae8de082b90250549891c45f4"}
Jul 06 19:11:39.191261 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:39.191240    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-j48g9" event={"ID":"1541f6e5-7700-4afb-8e59-7d03662da976","Type":"ContainerStarted","Data":"37d8807165610ab07a73c08184634ac01b23c59f015ba47bc338af34dff8b057"}
Jul 06 19:11:41.148375 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:11:41.148345    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:11:53.297254 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:11:53.297197    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"bbf6f300-9954-49ff-8236-e44e0c7dfde2\\\"},\\\"status\\\":{\\\"containerStatuses\\\":[{\\\"allocatedResources\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"64Mi\\\"},\\\"containerID\\\":\\\"cri-o://5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b\\\",\\\"image\\\":\\\"quay.io/opendatahub/odh-model-controller:odh-v3.5-EA2\\\",\\\"imageID\\\":\\\"quay.io/opendatahub/odh-model-controller@sha256:1e2397cbb77c598f2eaff55caa3f0ef39f9f499bc7388d4d92be8d2a5d5ff121\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-06T19:11:08Z\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-06T19:10:58Z\\\"}},\\\"name\\\":\\\"manager\\\",\\\"ready\\\":false,\\\"resources\\\":{\\\"limits\\\":{\\\"cpu\\\":\\\"500m\\\",\\\"memory\\\":\\\"2Gi\\\"},\\\"requests\\\":{\\\"cpu\\\":\\\"10m\\\",\\\"memory\\\":\\\"64Mi\\\"}},\\\"restartCount\\\":3,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-06T19:11:29Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000660000],\\\"uid\\\":1000660000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/tmp/k8s-webhook-server/serving-certs\\\",\\\"name\\\":\\\"cert\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-qpcvw\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"opendatahub\"/\"odh-model-controller-7998c67dc4-b88jv\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ed6006ff-308f-4519-b84f-e305d9418cee.svc:9743/pod?timeout=10s\": context deadline exceeded" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:12:03.318640 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:03.318586    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"55cbd3dc-9b9d-46b3-82c6-aa3221f970c7\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastTransitionTime\\\":\\\"2026-07-06T19:11:39Z\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastTransitionTime\\\":\\\"2026-07-06T19:11:39Z\\\",\\\"type\\\":\\\"ContainersReady\\\"}],\\\"containerStatuses\\\":[{\\\"containerID\\\":\\\"cri-o://9cdefd824a9f5151278746f53a601bb4c9595baae8de082b90250549891c45f4\\\",\\\"image\\\":\\\"registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:f3683d29aa703829fb0d0ba93711188fb717cc1ac156877c9d0ab5fb5083513f\\\",\\\"imageID\\\":\\\"registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:681a5f3e02131df294a2b83933584403fe4035d43d98530a13f83a77f71171fa\\\",\\\"lastState\\\":{\\\"terminated\\\":{\\\"containerID\\\":\\\"cri-o://f4b8605ecd9d84bf98a67727e4bbd9b8ad609c92dd665a21f6275419f4c5cb08\\\",\\\"exitCode\\\":1,\\\"finishedAt\\\":\\\"2026-07-06T19:11:37Z\\\",\\\"reason\\\":\\\"Error\\\",\\\"startedAt\\\":\\\"2026-07-06T19:05:15Z\\\"}},\\\"name\\\":\\\"cert-manager-cainjector\\\",\\\"ready\\\":true,\\\"resources\\\":{},\\\"restartCount\\\":1,\\\"started\\\":true,\\\"state\\\":{\\\"running\\\":{\\\"startedAt\\\":\\\"2026-07-06T19:11:38Z\\\"}},\\\"user\\\":{\\\"linux\\\":{\\\"gid\\\":0,\\\"supplementalGroups\\\":[0,1000640000],\\\"uid\\\":1000640000}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/openshift/serviceaccount\\\",\\\"name\\\":\\\"bound-sa-token\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"},{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-qnmb8\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}]}}\" for pod \"cert-manager\"/\"cert-manager-cainjector-559cdd8c87-rgbrm\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ed6006ff-308f-4519-b84f-e305d9418cee.svc:9743/pod?timeout=10s\": context deadline exceeded" pod="cert-manager/cert-manager-cainjector-559cdd8c87-rgbrm"
Jul 06 19:12:09.621551 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.621508    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7899bb5f99-jbjwc"]
Jul 06 19:12:09.625581 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.625558    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:09.650955 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.650929    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 06 19:12:09.732860 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.732834    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jtpr6\" (UniqueName: \"kubernetes.io/projected/4ad093a4-5e68-4a3e-91a7-c5d319954925-kube-api-access-jtpr6\") pod \"authorino-7899bb5f99-jbjwc\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") " pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:09.732977 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.732869    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ad093a4-5e68-4a3e-91a7-c5d319954925-tls-cert\") pod \"authorino-7899bb5f99-jbjwc\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") " pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:09.833650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.833626    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jtpr6\" (UniqueName: \"kubernetes.io/projected/4ad093a4-5e68-4a3e-91a7-c5d319954925-kube-api-access-jtpr6\") pod \"authorino-7899bb5f99-jbjwc\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") " pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:09.833747 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.833681    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ad093a4-5e68-4a3e-91a7-c5d319954925-tls-cert\") pod \"authorino-7899bb5f99-jbjwc\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") " pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:09.835931 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:09.835911    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ad093a4-5e68-4a3e-91a7-c5d319954925-tls-cert\") pod \"authorino-7899bb5f99-jbjwc\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") " pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:10.614234 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.614203    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pmrrm"]
Jul 06 19:12:10.617852 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.617830    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-558d9d87f9-z6zbh"]
Jul 06 19:12:10.617984 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.617967    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pmrrm"
Jul 06 19:12:10.621888 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.621869    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558d9d87f9-z6zbh"
Jul 06 19:12:10.740505 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.740482    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sltsh\" (UniqueName: \"kubernetes.io/projected/7bf559f5-d532-46dd-bfd3-6d80f924222a-kube-api-access-sltsh\") pod \"authorino-558d9d87f9-z6zbh\" (UID: \"7bf559f5-d532-46dd-bfd3-6d80f924222a\") " pod="kuadrant-system/authorino-558d9d87f9-z6zbh"
Jul 06 19:12:10.740624 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.740544    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbz6q\" (UniqueName: \"kubernetes.io/projected/873aa655-fa99-458c-b648-2d9d52f81689-kube-api-access-sbz6q\") pod \"authorino-5d4b67c446-pmrrm\" (UID: \"873aa655-fa99-458c-b648-2d9d52f81689\") " pod="kuadrant-system/authorino-5d4b67c446-pmrrm"
Jul 06 19:12:10.840996 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.840971    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbz6q\" (UniqueName: \"kubernetes.io/projected/873aa655-fa99-458c-b648-2d9d52f81689-kube-api-access-sbz6q\") pod \"authorino-5d4b67c446-pmrrm\" (UID: \"873aa655-fa99-458c-b648-2d9d52f81689\") " pod="kuadrant-system/authorino-5d4b67c446-pmrrm"
Jul 06 19:12:10.841088 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:10.841010    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sltsh\" (UniqueName: \"kubernetes.io/projected/7bf559f5-d532-46dd-bfd3-6d80f924222a-kube-api-access-sltsh\") pod \"authorino-558d9d87f9-z6zbh\" (UID: \"7bf559f5-d532-46dd-bfd3-6d80f924222a\") " pod="kuadrant-system/authorino-558d9d87f9-z6zbh"
Jul 06 19:12:11.187504 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:11.187472    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbz6q\" (UniqueName: \"kubernetes.io/projected/873aa655-fa99-458c-b648-2d9d52f81689-kube-api-access-sbz6q\") pod \"authorino-5d4b67c446-pmrrm\" (UID: \"873aa655-fa99-458c-b648-2d9d52f81689\") " pod="kuadrant-system/authorino-5d4b67c446-pmrrm"
Jul 06 19:12:11.200971 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:11.200941    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jtpr6\" (UniqueName: \"kubernetes.io/projected/4ad093a4-5e68-4a3e-91a7-c5d319954925-kube-api-access-jtpr6\") pod \"authorino-7899bb5f99-jbjwc\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") " pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:11.227963 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:11.227934    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pmrrm"
Jul 06 19:12:11.228906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:11.228886    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sltsh\" (UniqueName: \"kubernetes.io/projected/7bf559f5-d532-46dd-bfd3-6d80f924222a-kube-api-access-sltsh\") pod \"authorino-558d9d87f9-z6zbh\" (UID: \"7bf559f5-d532-46dd-bfd3-6d80f924222a\") " pod="kuadrant-system/authorino-558d9d87f9-z6zbh"
Jul 06 19:12:11.232885 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:11.232868    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558d9d87f9-z6zbh"
Jul 06 19:12:11.436222 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:11.436188    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:12:12.478585 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:12.478550    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pmrrm"]
Jul 06 19:12:13.340146 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:13.340118    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-558d9d87f9-z6zbh"]
Jul 06 19:12:27.092176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:27.091967    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7899bb5f99-jbjwc"]
Jul 06 19:12:30.430129 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.430096    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f2ac7c7-46df-4479-a0fc-bd3b25ad8787" containerID="166a5ab491a5cf8661e3409b4a5e0ab9d99f3dc2899afcb1cea677142b179587" exitCode=1
Jul 06 19:12:30.430550 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.430175    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" event={"ID":"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787","Type":"ContainerDied","Data":"166a5ab491a5cf8661e3409b4a5e0ab9d99f3dc2899afcb1cea677142b179587"}
Jul 06 19:12:30.430550 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.430249    2579 scope.go:117] "RemoveContainer" containerID="7e787a4efa7a8371964b7c516e254a4bd3b6ebe0652035d80930b74e6a042f35"
Jul 06 19:12:30.430694 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.430573    2579 scope.go:117] "RemoveContainer" containerID="166a5ab491a5cf8661e3409b4a5e0ab9d99f3dc2899afcb1cea677142b179587"
Jul 06 19:12:30.430803 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:30.430782    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=lws-controller-manager-84979c6fd9-757nm_openshift-lws-operator(1f2ac7c7-46df-4479-a0fc-bd3b25ad8787)\"" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" podUID="1f2ac7c7-46df-4479-a0fc-bd3b25ad8787"
Jul 06 19:12:30.432068 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.432043    2579 generic.go:356] "Generic (PLEG): container finished" podID="bbf6f300-9954-49ff-8236-e44e0c7dfde2" containerID="5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b" exitCode=1
Jul 06 19:12:30.432157 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.432081    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerDied","Data":"5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"}
Jul 06 19:12:30.432321 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.432309    2579 scope.go:117] "RemoveContainer" containerID="5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"
Jul 06 19:12:30.432455 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:30.432442    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:12:30.440855 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:30.440839    2579 scope.go:117] "RemoveContainer" containerID="80fe68ab81881211f3412480901997d72ccc17d482ed756049042191618f730b"
Jul 06 19:12:31.146175 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:31.146147    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:12:31.416903 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:31.416833    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-558d9d87f9-z6zbh"]
Jul 06 19:12:31.438351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:31.438327    2579 scope.go:117] "RemoveContainer" containerID="5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"
Jul 06 19:12:31.438757 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:31.438527    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:12:31.576834 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:31.576803    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pmrrm"]
Jul 06 19:12:31.657543 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:31.657508    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7899bb5f99-jbjwc"]
Jul 06 19:12:31.674417 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:12:31.674385    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ad093a4_5e68_4a3e_91a7_c5d319954925.slice/crio-bf121882c4e631662c1d9c8b3b8f8f4c62180c7e1d632aaf45c61aec6335a21a WatchSource:0}: Error finding container bf121882c4e631662c1d9c8b3b8f8f4c62180c7e1d632aaf45c61aec6335a21a: Status 404 returned error can't find the container with id bf121882c4e631662c1d9c8b3b8f8f4c62180c7e1d632aaf45c61aec6335a21a
Jul 06 19:12:31.711221 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:31.711197    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-558d9d87f9-z6zbh"]
Jul 06 19:12:31.712162 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:12:31.712131    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7bf559f5_d532_46dd_bfd3_6d80f924222a.slice/crio-fc8281c384ab844afe93b8d2a4026a8871af538b89f91e05ec7faa3f4fa7d987 WatchSource:0}: Error finding container fc8281c384ab844afe93b8d2a4026a8871af538b89f91e05ec7faa3f4fa7d987: Status 404 returned error can't find the container with id fc8281c384ab844afe93b8d2a4026a8871af538b89f91e05ec7faa3f4fa7d987
Jul 06 19:12:31.855685 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:31.855641    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pmrrm"]
Jul 06 19:12:31.857609 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:12:31.857586    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod873aa655_fa99_458c_b648_2d9d52f81689.slice/crio-f06ea1c845c84494fecb01ae28ce139c54f711a75251dca343d1a89f88724c1b WatchSource:0}: Error finding container f06ea1c845c84494fecb01ae28ce139c54f711a75251dca343d1a89f88724c1b: Status 404 returned error can't find the container with id f06ea1c845c84494fecb01ae28ce139c54f711a75251dca343d1a89f88724c1b
Jul 06 19:12:32.450843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.450803    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558d9d87f9-z6zbh" event={"ID":"7bf559f5-d532-46dd-bfd3-6d80f924222a","Type":"ContainerStarted","Data":"c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f"}
Jul 06 19:12:32.451345 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.451143    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558d9d87f9-z6zbh" event={"ID":"7bf559f5-d532-46dd-bfd3-6d80f924222a","Type":"ContainerStarted","Data":"fc8281c384ab844afe93b8d2a4026a8871af538b89f91e05ec7faa3f4fa7d987"}
Jul 06 19:12:32.451345 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.450945    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-558d9d87f9-z6zbh" podUID="7bf559f5-d532-46dd-bfd3-6d80f924222a" containerName="authorino" containerID="cri-o://c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f" gracePeriod=30
Jul 06 19:12:32.453207 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.453174    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-pmrrm" podUID="873aa655-fa99-458c-b648-2d9d52f81689" containerName="authorino" containerID="cri-o://33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7" gracePeriod=30
Jul 06 19:12:32.453331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.453215    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pmrrm" event={"ID":"873aa655-fa99-458c-b648-2d9d52f81689","Type":"ContainerStarted","Data":"33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7"}
Jul 06 19:12:32.453331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.453252    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pmrrm" event={"ID":"873aa655-fa99-458c-b648-2d9d52f81689","Type":"ContainerStarted","Data":"f06ea1c845c84494fecb01ae28ce139c54f711a75251dca343d1a89f88724c1b"}
Jul 06 19:12:32.455091 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.455066    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899bb5f99-jbjwc" event={"ID":"4ad093a4-5e68-4a3e-91a7-c5d319954925","Type":"ContainerStarted","Data":"58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f"}
Jul 06 19:12:32.455188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.455102    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899bb5f99-jbjwc" event={"ID":"4ad093a4-5e68-4a3e-91a7-c5d319954925","Type":"ContainerStarted","Data":"bf121882c4e631662c1d9c8b3b8f8f4c62180c7e1d632aaf45c61aec6335a21a"}
Jul 06 19:12:32.486450 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.486408    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-558d9d87f9-z6zbh" podStartSLOduration=24.923803005 podStartE2EDuration="25.486396007s" totalImagesPullingTime="562.593002ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:12:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:12:32.48490255 +0000 UTC m=+812.096228089" watchObservedRunningTime="2026-07-06 19:12:32.486396007 +0000 UTC m=+812.097721539"
Jul 06 19:12:32.532494 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.532450    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-pmrrm" podStartSLOduration=27.111530315 podStartE2EDuration="27.532434397s" totalImagesPullingTime="420.904082ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:12:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:12:32.530671925 +0000 UTC m=+812.141997454" watchObservedRunningTime="2026-07-06 19:12:32.532434397 +0000 UTC m=+812.143759935"
Jul 06 19:12:32.577603 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.577514    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7899bb5f99-jbjwc" podStartSLOduration=28.180092384 podStartE2EDuration="28.57749623s" totalImagesPullingTime="397.403846ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:12:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:12:32.572783855 +0000 UTC m=+812.184109393" watchObservedRunningTime="2026-07-06 19:12:32.57749623 +0000 UTC m=+812.188821768"
Jul 06 19:12:32.710313 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.710284    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mw2l"]
Jul 06 19:12:32.710486 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.710450    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-7mw2l" podUID="96b52bc7-a7a4-45c1-8d93-ec256f4b095b" containerName="authorino" containerID="cri-o://f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297" gracePeriod=30
Jul 06 19:12:32.800186 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.800159    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pmrrm"
Jul 06 19:12:32.803012 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.802991    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558d9d87f9-z6zbh"
Jul 06 19:12:32.901109 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.901083    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sltsh\" (UniqueName: \"kubernetes.io/projected/7bf559f5-d532-46dd-bfd3-6d80f924222a-kube-api-access-sltsh\") pod \"7bf559f5-d532-46dd-bfd3-6d80f924222a\" (UID: \"7bf559f5-d532-46dd-bfd3-6d80f924222a\") "
Jul 06 19:12:32.901283 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.901267    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sbz6q\" (UniqueName: \"kubernetes.io/projected/873aa655-fa99-458c-b648-2d9d52f81689-kube-api-access-sbz6q\") pod \"873aa655-fa99-458c-b648-2d9d52f81689\" (UID: \"873aa655-fa99-458c-b648-2d9d52f81689\") "
Jul 06 19:12:32.903258 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.903229    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/873aa655-fa99-458c-b648-2d9d52f81689-kube-api-access-sbz6q" (OuterVolumeSpecName: "kube-api-access-sbz6q") pod "873aa655-fa99-458c-b648-2d9d52f81689" (UID: "873aa655-fa99-458c-b648-2d9d52f81689"). InnerVolumeSpecName "kube-api-access-sbz6q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:12:32.903560 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.903541    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7bf559f5-d532-46dd-bfd3-6d80f924222a-kube-api-access-sltsh" (OuterVolumeSpecName: "kube-api-access-sltsh") pod "7bf559f5-d532-46dd-bfd3-6d80f924222a" (UID: "7bf559f5-d532-46dd-bfd3-6d80f924222a"). InnerVolumeSpecName "kube-api-access-sltsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:12:32.948309 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:32.948288    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mw2l"
Jul 06 19:12:33.002016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.001989    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sbz6q\" (UniqueName: \"kubernetes.io/projected/873aa655-fa99-458c-b648-2d9d52f81689-kube-api-access-sbz6q\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:12:33.002016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.002014    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sltsh\" (UniqueName: \"kubernetes.io/projected/7bf559f5-d532-46dd-bfd3-6d80f924222a-kube-api-access-sltsh\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:12:33.102903 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.102881    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dml4j\" (UniqueName: \"kubernetes.io/projected/96b52bc7-a7a4-45c1-8d93-ec256f4b095b-kube-api-access-dml4j\") pod \"96b52bc7-a7a4-45c1-8d93-ec256f4b095b\" (UID: \"96b52bc7-a7a4-45c1-8d93-ec256f4b095b\") "
Jul 06 19:12:33.104841 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.104819    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96b52bc7-a7a4-45c1-8d93-ec256f4b095b-kube-api-access-dml4j" (OuterVolumeSpecName: "kube-api-access-dml4j") pod "96b52bc7-a7a4-45c1-8d93-ec256f4b095b" (UID: "96b52bc7-a7a4-45c1-8d93-ec256f4b095b"). InnerVolumeSpecName "kube-api-access-dml4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:12:33.203492 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.203440    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dml4j\" (UniqueName: \"kubernetes.io/projected/96b52bc7-a7a4-45c1-8d93-ec256f4b095b-kube-api-access-dml4j\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:12:33.461020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.460944    2579 generic.go:356] "Generic (PLEG): container finished" podID="96b52bc7-a7a4-45c1-8d93-ec256f4b095b" containerID="f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297" exitCode=0
Jul 06 19:12:33.461020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.460986    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-7mw2l"
Jul 06 19:12:33.461483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.461038    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mw2l" event={"ID":"96b52bc7-a7a4-45c1-8d93-ec256f4b095b","Type":"ContainerDied","Data":"f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297"}
Jul 06 19:12:33.461483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.461097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-7mw2l" event={"ID":"96b52bc7-a7a4-45c1-8d93-ec256f4b095b","Type":"ContainerDied","Data":"b49eff67db52c9561a89259d5b1074a8019ba62872a6601d599b06999b61436f"}
Jul 06 19:12:33.461483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.461123    2579 scope.go:117] "RemoveContainer" containerID="f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297"
Jul 06 19:12:33.462429 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.462411    2579 generic.go:356] "Generic (PLEG): container finished" podID="7bf559f5-d532-46dd-bfd3-6d80f924222a" containerID="c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f" exitCode=0
Jul 06 19:12:33.462525 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.462461    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-558d9d87f9-z6zbh"
Jul 06 19:12:33.462525 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.462494    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558d9d87f9-z6zbh" event={"ID":"7bf559f5-d532-46dd-bfd3-6d80f924222a","Type":"ContainerDied","Data":"c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f"}
Jul 06 19:12:33.462634 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.462526    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-558d9d87f9-z6zbh" event={"ID":"7bf559f5-d532-46dd-bfd3-6d80f924222a","Type":"ContainerDied","Data":"fc8281c384ab844afe93b8d2a4026a8871af538b89f91e05ec7faa3f4fa7d987"}
Jul 06 19:12:33.464026 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.464002    2579 generic.go:356] "Generic (PLEG): container finished" podID="873aa655-fa99-458c-b648-2d9d52f81689" containerID="33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7" exitCode=0
Jul 06 19:12:33.464136 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.464039    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-pmrrm"
Jul 06 19:12:33.464199 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.464130    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pmrrm" event={"ID":"873aa655-fa99-458c-b648-2d9d52f81689","Type":"ContainerDied","Data":"33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7"}
Jul 06 19:12:33.464199 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.464167    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-pmrrm" event={"ID":"873aa655-fa99-458c-b648-2d9d52f81689","Type":"ContainerDied","Data":"f06ea1c845c84494fecb01ae28ce139c54f711a75251dca343d1a89f88724c1b"}
Jul 06 19:12:33.471133 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.471118    2579 scope.go:117] "RemoveContainer" containerID="f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297"
Jul 06 19:12:33.471349 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:33.471332    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297\": container with ID starting with f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297 not found: ID does not exist" containerID="f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297"
Jul 06 19:12:33.471397 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.471356    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297"} err="failed to get container status \"f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297\": rpc error: code = NotFound desc = could not find container \"f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297\": container with ID starting with f1ae9907b28f7185099110417692dfdb2d778097dde06fe6a9abfe384a000297 not found: ID does not exist"
Jul 06 19:12:33.471397 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.471372    2579 scope.go:117] "RemoveContainer" containerID="c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f"
Jul 06 19:12:33.478729 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.478714    2579 scope.go:117] "RemoveContainer" containerID="c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f"
Jul 06 19:12:33.478972 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:33.478947    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f\": container with ID starting with c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f not found: ID does not exist" containerID="c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f"
Jul 06 19:12:33.479055 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.478975    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f"} err="failed to get container status \"c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f\": rpc error: code = NotFound desc = could not find container \"c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f\": container with ID starting with c549032b19a0db2d7154711e4ffb76f006688da23cb549051ab29f8e314dc51f not found: ID does not exist"
Jul 06 19:12:33.479055 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.478990    2579 scope.go:117] "RemoveContainer" containerID="33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7"
Jul 06 19:12:33.485900 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.485883    2579 scope.go:117] "RemoveContainer" containerID="33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7"
Jul 06 19:12:33.486161 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:33.486146    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7\": container with ID starting with 33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7 not found: ID does not exist" containerID="33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7"
Jul 06 19:12:33.486216 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.486168    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7"} err="failed to get container status \"33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7\": rpc error: code = NotFound desc = could not find container \"33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7\": container with ID starting with 33b5d2d17bc0ca656597fc538b47dc0fea1dfbf1bba378d0d2a7fe51f7a88ee7 not found: ID does not exist"
Jul 06 19:12:33.529865 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.529840    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mw2l"]
Jul 06 19:12:33.538024 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.538003    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-7mw2l"]
Jul 06 19:12:33.575920 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.575898    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-558d9d87f9-z6zbh"]
Jul 06 19:12:33.591025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.591000    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-558d9d87f9-z6zbh"]
Jul 06 19:12:33.636846 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.636822    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pmrrm"]
Jul 06 19:12:33.647517 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:33.647494    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-pmrrm"]
Jul 06 19:12:34.482451 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:34.482424    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:12:34.482947 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:34.482766    2579 scope.go:117] "RemoveContainer" containerID="166a5ab491a5cf8661e3409b4a5e0ab9d99f3dc2899afcb1cea677142b179587"
Jul 06 19:12:34.482947 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:34.482912    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 10s restarting failed container=manager pod=lws-controller-manager-84979c6fd9-757nm_openshift-lws-operator(1f2ac7c7-46df-4479-a0fc-bd3b25ad8787)\"" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" podUID="1f2ac7c7-46df-4479-a0fc-bd3b25ad8787"
Jul 06 19:12:34.926608 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:34.926575    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7bf559f5-d532-46dd-bfd3-6d80f924222a" path="/var/lib/kubelet/pods/7bf559f5-d532-46dd-bfd3-6d80f924222a/volumes"
Jul 06 19:12:34.926909 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:34.926895    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="873aa655-fa99-458c-b648-2d9d52f81689" path="/var/lib/kubelet/pods/873aa655-fa99-458c-b648-2d9d52f81689/volumes"
Jul 06 19:12:34.927183 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:34.927172    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96b52bc7-a7a4-45c1-8d93-ec256f4b095b" path="/var/lib/kubelet/pods/96b52bc7-a7a4-45c1-8d93-ec256f4b095b/volumes"
Jul 06 19:12:37.598104 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:37.598069    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:12:37.598560 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:37.598506    2579 scope.go:117] "RemoveContainer" containerID="5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"
Jul 06 19:12:37.598765 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:37.598741    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:12:38.569005 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.568971    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7558657f96-hxmcg"]
Jul 06 19:12:38.569282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569269    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="873aa655-fa99-458c-b648-2d9d52f81689" containerName="authorino"
Jul 06 19:12:38.569325 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569284    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="873aa655-fa99-458c-b648-2d9d52f81689" containerName="authorino"
Jul 06 19:12:38.569325 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569297    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7bf559f5-d532-46dd-bfd3-6d80f924222a" containerName="authorino"
Jul 06 19:12:38.569325 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569305    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7bf559f5-d532-46dd-bfd3-6d80f924222a" containerName="authorino"
Jul 06 19:12:38.569325 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569323    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96b52bc7-a7a4-45c1-8d93-ec256f4b095b" containerName="authorino"
Jul 06 19:12:38.569451 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569329    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="96b52bc7-a7a4-45c1-8d93-ec256f4b095b" containerName="authorino"
Jul 06 19:12:38.569451 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569377    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="873aa655-fa99-458c-b648-2d9d52f81689" containerName="authorino"
Jul 06 19:12:38.569451 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569388    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="96b52bc7-a7a4-45c1-8d93-ec256f4b095b" containerName="authorino"
Jul 06 19:12:38.569451 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.569398    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7bf559f5-d532-46dd-bfd3-6d80f924222a" containerName="authorino"
Jul 06 19:12:38.573430 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.573413    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:38.583875 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.583858    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 19:12:38.584886 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.584872    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 19:12:38.626985 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.626955    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7558657f96-hxmcg"]
Jul 06 19:12:38.640145 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.640122    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpq72\" (UniqueName: \"kubernetes.io/projected/e1a585db-b4d0-4b80-9df3-75272ae8cece-kube-api-access-tpq72\") pod \"maas-controller-7558657f96-hxmcg\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") " pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:38.640252 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.640154    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1a585db-b4d0-4b80-9df3-75272ae8cece-webhook-cert\") pod \"maas-controller-7558657f96-hxmcg\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") " pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:38.741412 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.741386    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpq72\" (UniqueName: \"kubernetes.io/projected/e1a585db-b4d0-4b80-9df3-75272ae8cece-kube-api-access-tpq72\") pod \"maas-controller-7558657f96-hxmcg\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") " pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:38.741522 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.741419    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1a585db-b4d0-4b80-9df3-75272ae8cece-webhook-cert\") pod \"maas-controller-7558657f96-hxmcg\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") " pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:38.743840 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.743822    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1a585db-b4d0-4b80-9df3-75272ae8cece-webhook-cert\") pod \"maas-controller-7558657f96-hxmcg\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") " pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:38.779665 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.779637    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpq72\" (UniqueName: \"kubernetes.io/projected/e1a585db-b4d0-4b80-9df3-75272ae8cece-kube-api-access-tpq72\") pod \"maas-controller-7558657f96-hxmcg\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") " pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:38.883312 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:38.883254    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:39.039437 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:39.039404    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7558657f96-hxmcg"]
Jul 06 19:12:39.044004 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:12:39.043969    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1a585db_b4d0_4b80_9df3_75272ae8cece.slice/crio-e598a7ca4c9088ada45223fb45910a1fbbbc3859b0dc7f323fc6ab67a489e09f WatchSource:0}: Error finding container e598a7ca4c9088ada45223fb45910a1fbbbc3859b0dc7f323fc6ab67a489e09f: Status 404 returned error can't find the container with id e598a7ca4c9088ada45223fb45910a1fbbbc3859b0dc7f323fc6ab67a489e09f
Jul 06 19:12:39.501811 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:39.501774    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7558657f96-hxmcg" event={"ID":"e1a585db-b4d0-4b80-9df3-75272ae8cece","Type":"ContainerStarted","Data":"e598a7ca4c9088ada45223fb45910a1fbbbc3859b0dc7f323fc6ab67a489e09f"}
Jul 06 19:12:42.517722 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:42.517689    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7558657f96-hxmcg" event={"ID":"e1a585db-b4d0-4b80-9df3-75272ae8cece","Type":"ContainerStarted","Data":"5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946"}
Jul 06 19:12:42.518106 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:42.517924    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:44.692115 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:44.692086    2579 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:12:44.692480 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:44.692431    2579 scope.go:117] "RemoveContainer" containerID="166a5ab491a5cf8661e3409b4a5e0ab9d99f3dc2899afcb1cea677142b179587"
Jul 06 19:12:45.534559 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:45.534527    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm" event={"ID":"1f2ac7c7-46df-4479-a0fc-bd3b25ad8787","Type":"ContainerStarted","Data":"588a320932831cf4035c854d9c953c839607c9bcd23564e936d3b531b92e3139"}
Jul 06 19:12:45.534745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:45.534726    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:12:45.726291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:45.726232    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7558657f96-hxmcg" podStartSLOduration=5.23723449 podStartE2EDuration="7.726215661s" totalImagesPullingTime="2.488981171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:12:42.58820694 +0000 UTC m=+822.199532478" watchObservedRunningTime="2026-07-06 19:12:45.726215661 +0000 UTC m=+825.337541198"
Jul 06 19:12:48.922355 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:48.922326    2579 scope.go:117] "RemoveContainer" containerID="5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"
Jul 06 19:12:48.922739 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:48.922493    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:12:53.529546 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:53.529516    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:12:56.541485 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:56.541456    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-84979c6fd9-757nm"
Jul 06 19:12:59.922106 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:12:59.922073    2579 scope.go:117] "RemoveContainer" containerID="5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"
Jul 06 19:12:59.922575 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:12:59.922236    2579 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"manager\" with CrashLoopBackOff: \"back-off 40s restarting failed container=manager pod=odh-model-controller-7998c67dc4-b88jv_opendatahub(bbf6f300-9954-49ff-8236-e44e0c7dfde2)\"" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" podUID="bbf6f300-9954-49ff-8236-e44e0c7dfde2"
Jul 06 19:13:14.928014 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:14.922638    2579 scope.go:117] "RemoveContainer" containerID="5387bff8bdcee550e9361ac58ce5e4a491ce9739dd693746803cf920ececd87b"
Jul 06 19:13:15.414460 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.414422    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-94rq2"]
Jul 06 19:13:15.416638 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.416616    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.420882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.420863    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-gql7g\""
Jul 06 19:13:15.421090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.421074    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 19:13:15.441706 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.441684    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-94rq2"]
Jul 06 19:13:15.527391 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.527361    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2eb8e7-622d-436e-97a8-171df0afc050-plugins-config-volume\") pod \"payload-pre-processing-7964457696-94rq2\" (UID: \"ad2eb8e7-622d-436e-97a8-171df0afc050\") " pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.527539 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.527418    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlh87\" (UniqueName: \"kubernetes.io/projected/ad2eb8e7-622d-436e-97a8-171df0afc050-kube-api-access-vlh87\") pod \"payload-pre-processing-7964457696-94rq2\" (UID: \"ad2eb8e7-622d-436e-97a8-171df0afc050\") " pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.575442 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.575370    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ppprd"]
Jul 06 19:13:15.577743 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.577727    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.598949 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.598926    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ppprd"]
Jul 06 19:13:15.628171 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.628142    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2eb8e7-622d-436e-97a8-171df0afc050-plugins-config-volume\") pod \"payload-pre-processing-7964457696-94rq2\" (UID: \"ad2eb8e7-622d-436e-97a8-171df0afc050\") " pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.628284 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.628194    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlh87\" (UniqueName: \"kubernetes.io/projected/ad2eb8e7-622d-436e-97a8-171df0afc050-kube-api-access-vlh87\") pod \"payload-pre-processing-7964457696-94rq2\" (UID: \"ad2eb8e7-622d-436e-97a8-171df0afc050\") " pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.628827 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.628808    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ad2eb8e7-622d-436e-97a8-171df0afc050-plugins-config-volume\") pod \"payload-pre-processing-7964457696-94rq2\" (UID: \"ad2eb8e7-622d-436e-97a8-171df0afc050\") " pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.639607 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.639578    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlh87\" (UniqueName: \"kubernetes.io/projected/ad2eb8e7-622d-436e-97a8-171df0afc050-kube-api-access-vlh87\") pod \"payload-pre-processing-7964457696-94rq2\" (UID: \"ad2eb8e7-622d-436e-97a8-171df0afc050\") " pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.683453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.683430    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv" event={"ID":"bbf6f300-9954-49ff-8236-e44e0c7dfde2","Type":"ContainerStarted","Data":"1c31a3db5f91027992565dec3cb46d757e145ef8e449161eca3cdcf401e34ad2"}
Jul 06 19:13:15.683637 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.683620    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:13:15.726405 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.726367    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:15.729395 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.729377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkdck\" (UniqueName: \"kubernetes.io/projected/5d97bef7-0442-4148-975d-33e45d3b6572-kube-api-access-lkdck\") pod \"payload-processing-58f86c7798-ppprd\" (UID: \"5d97bef7-0442-4148-975d-33e45d3b6572\") " pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.729497 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.729428    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d97bef7-0442-4148-975d-33e45d3b6572-plugins-config-volume\") pod \"payload-processing-58f86c7798-ppprd\" (UID: \"5d97bef7-0442-4148-975d-33e45d3b6572\") " pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.829913 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.829829    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d97bef7-0442-4148-975d-33e45d3b6572-plugins-config-volume\") pod \"payload-processing-58f86c7798-ppprd\" (UID: \"5d97bef7-0442-4148-975d-33e45d3b6572\") " pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.830062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.829926    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkdck\" (UniqueName: \"kubernetes.io/projected/5d97bef7-0442-4148-975d-33e45d3b6572-kube-api-access-lkdck\") pod \"payload-processing-58f86c7798-ppprd\" (UID: \"5d97bef7-0442-4148-975d-33e45d3b6572\") " pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.830626 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.830604    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5d97bef7-0442-4148-975d-33e45d3b6572-plugins-config-volume\") pod \"payload-processing-58f86c7798-ppprd\" (UID: \"5d97bef7-0442-4148-975d-33e45d3b6572\") " pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.843442 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.843419    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkdck\" (UniqueName: \"kubernetes.io/projected/5d97bef7-0442-4148-975d-33e45d3b6572-kube-api-access-lkdck\") pod \"payload-processing-58f86c7798-ppprd\" (UID: \"5d97bef7-0442-4148-975d-33e45d3b6572\") " pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.887335 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.887302    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:15.897897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:15.897875    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-94rq2"]
Jul 06 19:13:15.900006 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:13:15.899971    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad2eb8e7_622d_436e_97a8_171df0afc050.slice/crio-7f1347e675d2a9357bf05ff8746021280b5568c5675f59324d2d6bffabb30568 WatchSource:0}: Error finding container 7f1347e675d2a9357bf05ff8746021280b5568c5675f59324d2d6bffabb30568: Status 404 returned error can't find the container with id 7f1347e675d2a9357bf05ff8746021280b5568c5675f59324d2d6bffabb30568
Jul 06 19:13:16.042693 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:16.042649    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-ppprd"]
Jul 06 19:13:16.045367 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:13:16.045339    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d97bef7_0442_4148_975d_33e45d3b6572.slice/crio-32a1cdafa884b9142cce2018efa7884c708986023f0110dde57021e5a397aa3e WatchSource:0}: Error finding container 32a1cdafa884b9142cce2018efa7884c708986023f0110dde57021e5a397aa3e: Status 404 returned error can't find the container with id 32a1cdafa884b9142cce2018efa7884c708986023f0110dde57021e5a397aa3e
Jul 06 19:13:16.689561 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:16.689523    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ppprd" event={"ID":"5d97bef7-0442-4148-975d-33e45d3b6572","Type":"ContainerStarted","Data":"32a1cdafa884b9142cce2018efa7884c708986023f0110dde57021e5a397aa3e"}
Jul 06 19:13:16.690689 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:16.690637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-94rq2" event={"ID":"ad2eb8e7-622d-436e-97a8-171df0afc050","Type":"ContainerStarted","Data":"7f1347e675d2a9357bf05ff8746021280b5568c5675f59324d2d6bffabb30568"}
Jul 06 19:13:17.697186 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:17.697097    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-ppprd" event={"ID":"5d97bef7-0442-4148-975d-33e45d3b6572","Type":"ContainerStarted","Data":"29dbb78bef27a1c66f74ff1a393833d25b5fc31711909c1c84f5b118e18f7f87"}
Jul 06 19:13:17.697591 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:17.697185    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:17.698674 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:17.698637    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-94rq2" event={"ID":"ad2eb8e7-622d-436e-97a8-171df0afc050","Type":"ContainerStarted","Data":"b9747816293835eed95d8bc1464ccb8aa43caaf322c7322c6ebc22594407cd40"}
Jul 06 19:13:17.698776 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:17.698740    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:13:17.722603 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:17.722561    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-ppprd" podStartSLOduration=1.421779088 podStartE2EDuration="2.722547971s" totalImagesPullingTime="1.300768883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:13:17.720745229 +0000 UTC m=+857.332070767" watchObservedRunningTime="2026-07-06 19:13:17.722547971 +0000 UTC m=+857.333873509"
Jul 06 19:13:17.760469 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:17.760425    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-94rq2" podStartSLOduration=1.325039037 podStartE2EDuration="2.760412137s" totalImagesPullingTime="1.4353731s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:13:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:13:17.759557154 +0000 UTC m=+857.370882694" watchObservedRunningTime="2026-07-06 19:13:17.760412137 +0000 UTC m=+857.371737674"
Jul 06 19:13:26.692943 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:26.692912    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-b88jv"
Jul 06 19:13:28.705322 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:28.705286    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-ppprd"
Jul 06 19:13:28.705754 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:13:28.705446    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-94rq2"
Jul 06 19:14:48.295675 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.295633    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"]
Jul 06 19:14:48.298986 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.298965    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.302259 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.302240    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kppft\""
Jul 06 19:14:48.302434 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.302420    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 19:14:48.303414 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.303394    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 19:14:48.303519 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.303437    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 19:14:48.317973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.317946    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"]
Jul 06 19:14:48.405201 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.405165    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.405348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.405217    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.405348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.405239    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e084384-b9cb-4e97-acb6-cff203045fc1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.405348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.405282    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.405348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.405314    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpcjg\" (UniqueName: \"kubernetes.io/projected/0e084384-b9cb-4e97-acb6-cff203045fc1-kube-api-access-bpcjg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.405509 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.405367    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506253 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506262    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506278    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e084384-b9cb-4e97-acb6-cff203045fc1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506294    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506322    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpcjg\" (UniqueName: \"kubernetes.io/projected/0e084384-b9cb-4e97-acb6-cff203045fc1-kube-api-access-bpcjg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506367    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506707 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506682    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506774 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506713    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.506813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.506796    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.508803 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.508781    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0e084384-b9cb-4e97-acb6-cff203045fc1-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.509241 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.509213    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0e084384-b9cb-4e97-acb6-cff203045fc1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.518090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.518071    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpcjg\" (UniqueName: \"kubernetes.io/projected/0e084384-b9cb-4e97-acb6-cff203045fc1-kube-api-access-bpcjg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq\" (UID: \"0e084384-b9cb-4e97-acb6-cff203045fc1\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.609153 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.609078    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:48.745151 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:48.745126    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"]
Jul 06 19:14:48.747272 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:14:48.747198    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e084384_b9cb_4e97_acb6_cff203045fc1.slice/crio-543a3c7a776e5f3d4a6037072967d6ee7a760ddd2b2f0f5a36922bca15cece32 WatchSource:0}: Error finding container 543a3c7a776e5f3d4a6037072967d6ee7a760ddd2b2f0f5a36922bca15cece32: Status 404 returned error can't find the container with id 543a3c7a776e5f3d4a6037072967d6ee7a760ddd2b2f0f5a36922bca15cece32
Jul 06 19:14:49.128788 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:49.128690    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq" event={"ID":"0e084384-b9cb-4e97-acb6-cff203045fc1","Type":"ContainerStarted","Data":"543a3c7a776e5f3d4a6037072967d6ee7a760ddd2b2f0f5a36922bca15cece32"}
Jul 06 19:14:50.135495 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:50.135413    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq" event={"ID":"0e084384-b9cb-4e97-acb6-cff203045fc1","Type":"ContainerStarted","Data":"fc8b179a8b6d43878d92539a25c6bb38c11f7ace2ee3f3ba078e2550abf11813"}
Jul 06 19:14:50.171866 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:50.171813    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq" podStartSLOduration=1.042415201 podStartE2EDuration="2.171794666s" totalImagesPullingTime="1.129379465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:14:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:14:50.169167058 +0000 UTC m=+949.780492595" watchObservedRunningTime="2026-07-06 19:14:50.171794666 +0000 UTC m=+949.783120205"
Jul 06 19:14:58.609856 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:58.609818    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:58.610308 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:58.609881    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:58.622388 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:58.622360    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:14:59.189779 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:14:59.189749    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq"
Jul 06 19:17:24.525510 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:17:24.525455    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7558657f96-hxmcg" podUID="e1a585db-b4d0-4b80-9df3-75272ae8cece" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.37:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:17:34.525586 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:17:34.525549    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7558657f96-hxmcg" podUID="e1a585db-b4d0-4b80-9df3-75272ae8cece" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.37:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:18:34.943935 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:34.943862    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5cb5dc74cd-2hz4v"]
Jul 06 19:18:34.947169 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:34.947149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:34.970567 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:34.970545    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb5dc74cd-2hz4v"]
Jul 06 19:18:35.092554 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.092522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef-tls-cert\") pod \"authorino-5cb5dc74cd-2hz4v\" (UID: \"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef\") " pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:35.092730 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.092575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sccr6\" (UniqueName: \"kubernetes.io/projected/45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef-kube-api-access-sccr6\") pod \"authorino-5cb5dc74cd-2hz4v\" (UID: \"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef\") " pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:35.193863 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.193835    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sccr6\" (UniqueName: \"kubernetes.io/projected/45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef-kube-api-access-sccr6\") pod \"authorino-5cb5dc74cd-2hz4v\" (UID: \"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef\") " pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:35.194006 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.193895    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef-tls-cert\") pod \"authorino-5cb5dc74cd-2hz4v\" (UID: \"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef\") " pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:35.196324 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.196300    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef-tls-cert\") pod \"authorino-5cb5dc74cd-2hz4v\" (UID: \"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef\") " pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:35.205677 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.205641    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sccr6\" (UniqueName: \"kubernetes.io/projected/45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef-kube-api-access-sccr6\") pod \"authorino-5cb5dc74cd-2hz4v\" (UID: \"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef\") " pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:35.255924 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.255905    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v"
Jul 06 19:18:35.395246 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.395218    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5cb5dc74cd-2hz4v"]
Jul 06 19:18:35.398676 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:18:35.398640    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45d2a523_0ce6_4f3b_bdfe_9ff157f1f5ef.slice/crio-cc73c031cd8a63df8f86e0ffe4e5291f77dec242aee5bfe58d09e2599fc120d4 WatchSource:0}: Error finding container cc73c031cd8a63df8f86e0ffe4e5291f77dec242aee5bfe58d09e2599fc120d4: Status 404 returned error can't find the container with id cc73c031cd8a63df8f86e0ffe4e5291f77dec242aee5bfe58d09e2599fc120d4
Jul 06 19:18:35.399879 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:35.399864    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:18:36.238527 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.238481    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v" event={"ID":"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef","Type":"ContainerStarted","Data":"a2fca2c4777d037a19aaa74c610bb5e425e713527e20d1f4b436c9ae643f4e0e"}
Jul 06 19:18:36.238527 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.238533    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v" event={"ID":"45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef","Type":"ContainerStarted","Data":"cc73c031cd8a63df8f86e0ffe4e5291f77dec242aee5bfe58d09e2599fc120d4"}
Jul 06 19:18:36.283667 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.283609    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5cb5dc74cd-2hz4v" podStartSLOduration=1.9275630110000002 podStartE2EDuration="2.283592813s" totalImagesPullingTime="356.029802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:18:36.281805515 +0000 UTC m=+1175.893131053" watchObservedRunningTime="2026-07-06 19:18:36.283592813 +0000 UTC m=+1175.894918350"
Jul 06 19:18:36.390824 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.390784    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7899bb5f99-jbjwc"]
Jul 06 19:18:36.390987 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.390965    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-7899bb5f99-jbjwc" podUID="4ad093a4-5e68-4a3e-91a7-c5d319954925" containerName="authorino" containerID="cri-o://58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f" gracePeriod=30
Jul 06 19:18:36.650380 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.650359    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:18:36.806741 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.806713    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ad093a4-5e68-4a3e-91a7-c5d319954925-tls-cert\") pod \"4ad093a4-5e68-4a3e-91a7-c5d319954925\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") "
Jul 06 19:18:36.806902 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.806855    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jtpr6\" (UniqueName: \"kubernetes.io/projected/4ad093a4-5e68-4a3e-91a7-c5d319954925-kube-api-access-jtpr6\") pod \"4ad093a4-5e68-4a3e-91a7-c5d319954925\" (UID: \"4ad093a4-5e68-4a3e-91a7-c5d319954925\") "
Jul 06 19:18:36.809099 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.809066    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad093a4-5e68-4a3e-91a7-c5d319954925-kube-api-access-jtpr6" (OuterVolumeSpecName: "kube-api-access-jtpr6") pod "4ad093a4-5e68-4a3e-91a7-c5d319954925" (UID: "4ad093a4-5e68-4a3e-91a7-c5d319954925"). InnerVolumeSpecName "kube-api-access-jtpr6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:18:36.816069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.816047    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ad093a4-5e68-4a3e-91a7-c5d319954925-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "4ad093a4-5e68-4a3e-91a7-c5d319954925" (UID: "4ad093a4-5e68-4a3e-91a7-c5d319954925"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:18:36.907649 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.907620    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jtpr6\" (UniqueName: \"kubernetes.io/projected/4ad093a4-5e68-4a3e-91a7-c5d319954925-kube-api-access-jtpr6\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:18:36.907649 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:36.907646    2579 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/4ad093a4-5e68-4a3e-91a7-c5d319954925-tls-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:18:37.244893 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.244860    2579 generic.go:356] "Generic (PLEG): container finished" podID="4ad093a4-5e68-4a3e-91a7-c5d319954925" containerID="58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f" exitCode=0
Jul 06 19:18:37.245253 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.244921    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7899bb5f99-jbjwc"
Jul 06 19:18:37.245253 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.244952    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899bb5f99-jbjwc" event={"ID":"4ad093a4-5e68-4a3e-91a7-c5d319954925","Type":"ContainerDied","Data":"58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f"}
Jul 06 19:18:37.245253 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.244989    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7899bb5f99-jbjwc" event={"ID":"4ad093a4-5e68-4a3e-91a7-c5d319954925","Type":"ContainerDied","Data":"bf121882c4e631662c1d9c8b3b8f8f4c62180c7e1d632aaf45c61aec6335a21a"}
Jul 06 19:18:37.245253 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.245008    2579 scope.go:117] "RemoveContainer" containerID="58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f"
Jul 06 19:18:37.254299 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.254282    2579 scope.go:117] "RemoveContainer" containerID="58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f"
Jul 06 19:18:37.254567 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:18:37.254548    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f\": container with ID starting with 58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f not found: ID does not exist" containerID="58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f"
Jul 06 19:18:37.254635 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.254581    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f"} err="failed to get container status \"58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f\": rpc error: code = NotFound desc = could not find container \"58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f\": container with ID starting with 58095e7dd6281d8d8af26ee52495a65b0508e7bc5ff4568d6e96c93c6ef4b22f not found: ID does not exist"
Jul 06 19:18:37.313920 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.313890    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7899bb5f99-jbjwc"]
Jul 06 19:18:37.335022 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:37.334997    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7899bb5f99-jbjwc"]
Jul 06 19:18:38.927206 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:18:38.927165    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ad093a4-5e68-4a3e-91a7-c5d319954925" path="/var/lib/kubelet/pods/4ad093a4-5e68-4a3e-91a7-c5d319954925/volumes"
Jul 06 19:19:05.496612 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.496577    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7558657f96-hxmcg"]
Jul 06 19:19:05.497073 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.496842    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7558657f96-hxmcg" podUID="e1a585db-b4d0-4b80-9df3-75272ae8cece" containerName="manager" containerID="cri-o://5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946" gracePeriod=30
Jul 06 19:19:05.748932 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.748880    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:19:05.812029 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.812000    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1a585db-b4d0-4b80-9df3-75272ae8cece-webhook-cert\") pod \"e1a585db-b4d0-4b80-9df3-75272ae8cece\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") "
Jul 06 19:19:05.812196 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.812071    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpq72\" (UniqueName: \"kubernetes.io/projected/e1a585db-b4d0-4b80-9df3-75272ae8cece-kube-api-access-tpq72\") pod \"e1a585db-b4d0-4b80-9df3-75272ae8cece\" (UID: \"e1a585db-b4d0-4b80-9df3-75272ae8cece\") "
Jul 06 19:19:05.814103 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.814071    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1a585db-b4d0-4b80-9df3-75272ae8cece-kube-api-access-tpq72" (OuterVolumeSpecName: "kube-api-access-tpq72") pod "e1a585db-b4d0-4b80-9df3-75272ae8cece" (UID: "e1a585db-b4d0-4b80-9df3-75272ae8cece"). InnerVolumeSpecName "kube-api-access-tpq72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:19:05.814208 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.814129    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1a585db-b4d0-4b80-9df3-75272ae8cece-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e1a585db-b4d0-4b80-9df3-75272ae8cece" (UID: "e1a585db-b4d0-4b80-9df3-75272ae8cece"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:19:05.913295 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.913270    2579 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e1a585db-b4d0-4b80-9df3-75272ae8cece-webhook-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:19:05.913295 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:05.913294    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpq72\" (UniqueName: \"kubernetes.io/projected/e1a585db-b4d0-4b80-9df3-75272ae8cece-kube-api-access-tpq72\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:19:06.404560 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.404528    2579 generic.go:356] "Generic (PLEG): container finished" podID="e1a585db-b4d0-4b80-9df3-75272ae8cece" containerID="5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946" exitCode=0
Jul 06 19:19:06.404825 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.404580    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7558657f96-hxmcg"
Jul 06 19:19:06.404825 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.404604    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7558657f96-hxmcg" event={"ID":"e1a585db-b4d0-4b80-9df3-75272ae8cece","Type":"ContainerDied","Data":"5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946"}
Jul 06 19:19:06.404825 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.404645    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7558657f96-hxmcg" event={"ID":"e1a585db-b4d0-4b80-9df3-75272ae8cece","Type":"ContainerDied","Data":"e598a7ca4c9088ada45223fb45910a1fbbbc3859b0dc7f323fc6ab67a489e09f"}
Jul 06 19:19:06.404825 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.404686    2579 scope.go:117] "RemoveContainer" containerID="5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946"
Jul 06 19:19:06.413608 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.413591    2579 scope.go:117] "RemoveContainer" containerID="5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946"
Jul 06 19:19:06.413880 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:19:06.413859    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946\": container with ID starting with 5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946 not found: ID does not exist" containerID="5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946"
Jul 06 19:19:06.413938 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.413887    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946"} err="failed to get container status \"5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946\": rpc error: code = NotFound desc = could not find container \"5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946\": container with ID starting with 5b6ad80bb02945b809359a0d907723513401298e1668eb9088b68ab4a7ebe946 not found: ID does not exist"
Jul 06 19:19:06.445089 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.445063    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7558657f96-hxmcg"]
Jul 06 19:19:06.453740 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.453723    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7558657f96-hxmcg"]
Jul 06 19:19:06.927359 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:06.927328    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1a585db-b4d0-4b80-9df3-75272ae8cece" path="/var/lib/kubelet/pods/e1a585db-b4d0-4b80-9df3-75272ae8cece/volumes"
Jul 06 19:19:07.406007 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.405971    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7558657f96-wwrjg"]
Jul 06 19:19:07.406306 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.406293    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ad093a4-5e68-4a3e-91a7-c5d319954925" containerName="authorino"
Jul 06 19:19:07.406353 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.406308    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ad093a4-5e68-4a3e-91a7-c5d319954925" containerName="authorino"
Jul 06 19:19:07.406353 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.406325    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1a585db-b4d0-4b80-9df3-75272ae8cece" containerName="manager"
Jul 06 19:19:07.406353 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.406331    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1a585db-b4d0-4b80-9df3-75272ae8cece" containerName="manager"
Jul 06 19:19:07.406446 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.406392    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1a585db-b4d0-4b80-9df3-75272ae8cece" containerName="manager"
Jul 06 19:19:07.406446 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.406402    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ad093a4-5e68-4a3e-91a7-c5d319954925" containerName="authorino"
Jul 06 19:19:07.410714 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.410695    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.423230 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.423210    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/457f7f2d-3135-4e8c-9fcd-d8d0641a8314-webhook-cert\") pod \"maas-controller-7558657f96-wwrjg\" (UID: \"457f7f2d-3135-4e8c-9fcd-d8d0641a8314\") " pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.423325 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.423247    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8pfq\" (UniqueName: \"kubernetes.io/projected/457f7f2d-3135-4e8c-9fcd-d8d0641a8314-kube-api-access-t8pfq\") pod \"maas-controller-7558657f96-wwrjg\" (UID: \"457f7f2d-3135-4e8c-9fcd-d8d0641a8314\") " pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.430753 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.430737    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-5m44x\""
Jul 06 19:19:07.435784 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.435768    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 19:19:07.445956 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.445938    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 19:19:07.467628 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.467608    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7558657f96-wwrjg"]
Jul 06 19:19:07.523712 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.523691    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/457f7f2d-3135-4e8c-9fcd-d8d0641a8314-webhook-cert\") pod \"maas-controller-7558657f96-wwrjg\" (UID: \"457f7f2d-3135-4e8c-9fcd-d8d0641a8314\") " pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.523840 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.523738    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8pfq\" (UniqueName: \"kubernetes.io/projected/457f7f2d-3135-4e8c-9fcd-d8d0641a8314-kube-api-access-t8pfq\") pod \"maas-controller-7558657f96-wwrjg\" (UID: \"457f7f2d-3135-4e8c-9fcd-d8d0641a8314\") " pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.526016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.525998    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/457f7f2d-3135-4e8c-9fcd-d8d0641a8314-webhook-cert\") pod \"maas-controller-7558657f96-wwrjg\" (UID: \"457f7f2d-3135-4e8c-9fcd-d8d0641a8314\") " pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.547070 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.547039    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8pfq\" (UniqueName: \"kubernetes.io/projected/457f7f2d-3135-4e8c-9fcd-d8d0641a8314-kube-api-access-t8pfq\") pod \"maas-controller-7558657f96-wwrjg\" (UID: \"457f7f2d-3135-4e8c-9fcd-d8d0641a8314\") " pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.720199 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.720143    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:07.866987 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:07.866960    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7558657f96-wwrjg"]
Jul 06 19:19:07.868963 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:19:07.868938    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod457f7f2d_3135_4e8c_9fcd_d8d0641a8314.slice/crio-9313c18a58082d7fade046d0f3f0e43a2f67aff7ea272b17fba80c1916d2b96e WatchSource:0}: Error finding container 9313c18a58082d7fade046d0f3f0e43a2f67aff7ea272b17fba80c1916d2b96e: Status 404 returned error can't find the container with id 9313c18a58082d7fade046d0f3f0e43a2f67aff7ea272b17fba80c1916d2b96e
Jul 06 19:19:08.421755 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:08.421721    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7558657f96-wwrjg" event={"ID":"457f7f2d-3135-4e8c-9fcd-d8d0641a8314","Type":"ContainerStarted","Data":"9313c18a58082d7fade046d0f3f0e43a2f67aff7ea272b17fba80c1916d2b96e"}
Jul 06 19:19:09.427365 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:09.427330    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7558657f96-wwrjg" event={"ID":"457f7f2d-3135-4e8c-9fcd-d8d0641a8314","Type":"ContainerStarted","Data":"578b1ded9ba6a0f0eaa9e644c88b879dfad0e2d683745abefc5f4975b881486a"}
Jul 06 19:19:09.427771 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:09.427389    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:19:09.469029 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:09.468981    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7558657f96-wwrjg" podStartSLOduration=1.764894652 podStartE2EDuration="2.468966278s" totalImagesPullingTime="704.071626ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:19:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:19:09.466997196 +0000 UTC m=+1209.078322753" watchObservedRunningTime="2026-07-06 19:19:09.468966278 +0000 UTC m=+1209.080291826"
Jul 06 19:19:20.437420 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:19:20.437383    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7558657f96-wwrjg"
Jul 06 19:21:31.433903 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:21:31.433811    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7558657f96-wwrjg" podUID="457f7f2d-3135-4e8c-9fcd-d8d0641a8314" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.42:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:30:00.178584 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:00.178551    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722770-z2dzr"]
Jul 06 19:30:00.181339 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:00.181322    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:00.184941 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:00.184914    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\""
Jul 06 19:30:00.333731 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:00.333697    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77m42\" (UniqueName: \"kubernetes.io/projected/ee97f373-cd4a-4d37-8bcf-e3328d0350ea-kube-api-access-77m42\") pod \"maas-api-key-cleanup-29722770-z2dzr\" (UID: \"ee97f373-cd4a-4d37-8bcf-e3328d0350ea\") " pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:00.434901 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:00.434826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77m42\" (UniqueName: \"kubernetes.io/projected/ee97f373-cd4a-4d37-8bcf-e3328d0350ea-kube-api-access-77m42\") pod \"maas-api-key-cleanup-29722770-z2dzr\" (UID: \"ee97f373-cd4a-4d37-8bcf-e3328d0350ea\") " pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:00.445723 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:00.445690    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-77m42\" (UniqueName: \"kubernetes.io/projected/ee97f373-cd4a-4d37-8bcf-e3328d0350ea-kube-api-access-77m42\") pod \"maas-api-key-cleanup-29722770-z2dzr\" (UID: \"ee97f373-cd4a-4d37-8bcf-e3328d0350ea\") " pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:00.494575 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:00.494550    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:10.195343 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:30:10.195292    2579 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ee97f373-cd4a-4d37-8bcf-e3328d0350ea\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-06T19:30:00Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-06T19:30:00Z\\\",\\\"observedGeneration\\\":1,\\\"status\\\":\\\"True\\\",\\\"type\\\":\\\"Initialized\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-06T19:30:00Z\\\",\\\"message\\\":\\\"containers with unready status: [cleanup]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"Ready\\\"},{\\\"lastProbeTime\\\":null,\\\"lastTransitionTime\\\":\\\"2026-07-06T19:30:00Z\\\",\\\"message\\\":\\\"containers with unready status: [cleanup]\\\",\\\"observedGeneration\\\":1,\\\"reason\\\":\\\"ContainersNotReady\\\",\\\"status\\\":\\\"False\\\",\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":1,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"containerStatuses\\\":[{\\\"image\\\":\\\"registry.redhat.io/ubi9/ubi-minimal:9.7\\\",\\\"imageID\\\":\\\"\\\",\\\"lastState\\\":{},\\\"name\\\":\\\"cleanup\\\",\\\"ready\\\":false,\\\"restartCount\\\":0,\\\"started\\\":false,\\\"state\\\":{\\\"waiting\\\":{\\\"reason\\\":\\\"ContainerCreating\\\"}},\\\"volumeMounts\\\":[{\\\"mountPath\\\":\\\"/var/run/secrets/kubernetes.io/serviceaccount\\\",\\\"name\\\":\\\"kube-api-access-77m42\\\",\\\"readOnly\\\":true,\\\"recursiveReadOnly\\\":\\\"Disabled\\\"}]}],\\\"hostIP\\\":\\\"10.0.142.165\\\",\\\"hostIPs\\\":[{\\\"ip\\\":\\\"10.0.142.165\\\"}],\\\"observedGeneration\\\":1,\\\"startTime\\\":\\\"2026-07-06T19:30:00Z\\\"}}\" for pod \"opendatahub\"/\"maas-api-key-cleanup-29722770-z2dzr\": Internal error occurred: failed calling webhook \"pod.network-node-identity.openshift.io\": failed to call webhook: Post \"https://network-node-identity.clusters-ed6006ff-308f-4519-b84f-e305d9418cee.svc:9743/pod?timeout=10s\": context deadline exceeded" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:10.249688 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:10.249634    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722770-z2dzr"]
Jul 06 19:30:10.472949 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:10.472921    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722770-z2dzr"]
Jul 06 19:30:10.474330 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:30:10.474296    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee97f373_cd4a_4d37_8bcf_e3328d0350ea.slice/crio-7a11a8e84c800ae24d811f3714c970f79b1015906eb783ce06f308add50c160f WatchSource:0}: Error finding container 7a11a8e84c800ae24d811f3714c970f79b1015906eb783ce06f308add50c160f: Status 404 returned error can't find the container with id 7a11a8e84c800ae24d811f3714c970f79b1015906eb783ce06f308add50c160f
Jul 06 19:30:10.476227 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:10.476208    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:30:10.619409 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:10.619376    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr" event={"ID":"ee97f373-cd4a-4d37-8bcf-e3328d0350ea","Type":"ContainerStarted","Data":"7a11a8e84c800ae24d811f3714c970f79b1015906eb783ce06f308add50c160f"}
Jul 06 19:30:18.659563 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:18.659532    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr" event={"ID":"ee97f373-cd4a-4d37-8bcf-e3328d0350ea","Type":"ContainerStarted","Data":"7d834340de5234caa0b674abc8bc5a7418ea31c41488ba0274cf57c09d32875a"}
Jul 06 19:30:18.684423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:18.684378    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr" podStartSLOduration=10.84256795 podStartE2EDuration="18.684365372s" totalImagesPullingTime="7.841797422s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:30:18.683316372 +0000 UTC m=+1878.294641911" watchObservedRunningTime="2026-07-06 19:30:18.684365372 +0000 UTC m=+1878.295690910"
Jul 06 19:30:19.665726 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:19.665693    2579 generic.go:356] "Generic (PLEG): container finished" podID="ee97f373-cd4a-4d37-8bcf-e3328d0350ea" containerID="7d834340de5234caa0b674abc8bc5a7418ea31c41488ba0274cf57c09d32875a" exitCode=0
Jul 06 19:30:19.666104 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:19.665733    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr" event={"ID":"ee97f373-cd4a-4d37-8bcf-e3328d0350ea","Type":"ContainerDied","Data":"7d834340de5234caa0b674abc8bc5a7418ea31c41488ba0274cf57c09d32875a"}
Jul 06 19:30:20.795097 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:20.795075    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:20.910794 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:20.910764    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77m42\" (UniqueName: \"kubernetes.io/projected/ee97f373-cd4a-4d37-8bcf-e3328d0350ea-kube-api-access-77m42\") pod \"ee97f373-cd4a-4d37-8bcf-e3328d0350ea\" (UID: \"ee97f373-cd4a-4d37-8bcf-e3328d0350ea\") "
Jul 06 19:30:20.912757 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:20.912732    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee97f373-cd4a-4d37-8bcf-e3328d0350ea-kube-api-access-77m42" (OuterVolumeSpecName: "kube-api-access-77m42") pod "ee97f373-cd4a-4d37-8bcf-e3328d0350ea" (UID: "ee97f373-cd4a-4d37-8bcf-e3328d0350ea"). InnerVolumeSpecName "kube-api-access-77m42". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:30:21.011757 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:21.011724    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77m42\" (UniqueName: \"kubernetes.io/projected/ee97f373-cd4a-4d37-8bcf-e3328d0350ea-kube-api-access-77m42\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:30:21.676905 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:21.676871    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr"
Jul 06 19:30:21.677062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:21.676900    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722770-z2dzr" event={"ID":"ee97f373-cd4a-4d37-8bcf-e3328d0350ea","Type":"ContainerDied","Data":"7a11a8e84c800ae24d811f3714c970f79b1015906eb783ce06f308add50c160f"}
Jul 06 19:30:21.677062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:30:21.676975    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7a11a8e84c800ae24d811f3714c970f79b1015906eb783ce06f308add50c160f"
Jul 06 19:31:51.368899 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.368862    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"]
Jul 06 19:31:51.369345 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.369305    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee97f373-cd4a-4d37-8bcf-e3328d0350ea" containerName="cleanup"
Jul 06 19:31:51.369345 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.369320    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee97f373-cd4a-4d37-8bcf-e3328d0350ea" containerName="cleanup"
Jul 06 19:31:51.369428 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.369410    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee97f373-cd4a-4d37-8bcf-e3328d0350ea" containerName="cleanup"
Jul 06 19:31:51.372435 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.372410    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.382486 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.382468    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 19:31:51.382615 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.382567    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a711c31a-openshift-default-dockercfg-lskjr\""
Jul 06 19:31:51.421391 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421355    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-certs\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421549 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421403    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-credential-socket\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421549 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-socket\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421549 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-envoy\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421549 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421514    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zztwq\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-kube-api-access-zztwq\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421549 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421532    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-data\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421734 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421596    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b1753a-5832-4bf4-82e2-7560709c8d80-istiod-ca-cert\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421734 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421618    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-podinfo\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.421734 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.421643    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-token\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.478889 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.478854    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"]
Jul 06 19:31:51.522694 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b1753a-5832-4bf4-82e2-7560709c8d80-istiod-ca-cert\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.522694 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522624    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-podinfo\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.522694 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522675    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-token\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.522694 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522696    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-certs\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522719    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-credential-socket\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522746    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-socket\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522765    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-envoy\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522900    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zztwq\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-kube-api-access-zztwq\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523015 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.522942    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-data\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523255 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.523118    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-credential-socket\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523255 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.523138    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-certs\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523255 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.523194    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-socket\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523418 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.523292    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-data\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.523760 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.523738    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b1753a-5832-4bf4-82e2-7560709c8d80-istiod-ca-cert\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.524981 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.524963    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-envoy\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.525207 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.525190    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-podinfo\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.556086 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.556055    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zztwq\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-kube-api-access-zztwq\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.565385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.565362    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-token\") pod \"e2e-ait-a711c31a-openshift-default-69f6495475-slr96\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") " pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.683594 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.683551    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:51.840588 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:51.840556    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"]
Jul 06 19:31:51.843871 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:31:51.843845    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b1753a_5832_4bf4_82e2_7560709c8d80.slice/crio-7c1c066138aa0950c28845f5e6abaff00cd2bb3eaa343c859d7c5fdf758e776b WatchSource:0}: Error finding container 7c1c066138aa0950c28845f5e6abaff00cd2bb3eaa343c859d7c5fdf758e776b: Status 404 returned error can't find the container with id 7c1c066138aa0950c28845f5e6abaff00cd2bb3eaa343c859d7c5fdf758e776b
Jul 06 19:31:52.122160 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:52.122074    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96" event={"ID":"76b1753a-5832-4bf4-82e2-7560709c8d80","Type":"ContainerStarted","Data":"7c1c066138aa0950c28845f5e6abaff00cd2bb3eaa343c859d7c5fdf758e776b"}
Jul 06 19:31:54.204213 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:54.204169    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:31:54.204490 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:54.204247    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:31:54.204490 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:54.204276    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:31:55.141066 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:55.141029    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96" event={"ID":"76b1753a-5832-4bf4-82e2-7560709c8d80","Type":"ContainerStarted","Data":"19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b"}
Jul 06 19:31:55.225134 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:55.225082    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96" podStartSLOduration=1.8669832259999999 podStartE2EDuration="4.225064905s" totalImagesPullingTime="2.358081679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:31:55.223345219 +0000 UTC m=+1974.834670757" watchObservedRunningTime="2026-07-06 19:31:55.225064905 +0000 UTC m=+1974.836390442"
Jul 06 19:31:55.684099 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:55.684064    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:55.688899 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:55.688871    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:56.145695 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:56.145667    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:56.146484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:56.146466    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:31:57.684329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:57.684300    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"]
Jul 06 19:31:58.154735 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:58.154703    2579 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96" secret="" err="secret \"e2e-ait-a711c31a-openshift-default-dockercfg-lskjr\" not found"
Jul 06 19:31:59.159663 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:31:59.159624    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96" podUID="76b1753a-5832-4bf4-82e2-7560709c8d80" containerName="istio-proxy" containerID="cri-o://19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b" gracePeriod=30
Jul 06 19:32:04.401562 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.401537    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:32:04.430565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430505    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-data\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430683 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430559    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-token\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430683 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430607    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-credential-socket\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430683 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430632    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b1753a-5832-4bf4-82e2-7560709c8d80-istiod-ca-cert\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430683 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430666    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-certs\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-socket\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430717    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zztwq\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-kube-api-access-zztwq\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430753    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-podinfo\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.430882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430792    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-envoy\") pod \"76b1753a-5832-4bf4-82e2-7560709c8d80\" (UID: \"76b1753a-5832-4bf4-82e2-7560709c8d80\") "
Jul 06 19:32:04.431087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430865    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-data" (OuterVolumeSpecName: "istio-data") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:04.431087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.430939    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:04.431087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.431060    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:04.431087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.431077    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76b1753a-5832-4bf4-82e2-7560709c8d80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:32:04.431318 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.431116    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.431318 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.431134    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.431318 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.431293    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:04.433116 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.433082    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:04.433369 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.433349    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:32:04.433439 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.433382    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-token" (OuterVolumeSpecName: "istio-token") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:32:04.433522 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.433501    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-kube-api-access-zztwq" (OuterVolumeSpecName: "kube-api-access-zztwq") pod "76b1753a-5832-4bf4-82e2-7560709c8d80" (UID: "76b1753a-5832-4bf4-82e2-7560709c8d80"). InnerVolumeSpecName "kube-api-access-zztwq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:32:04.531622 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.531596    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.531622 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.531621    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76b1753a-5832-4bf4-82e2-7560709c8d80-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.531812 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.531635    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.531812 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.531646    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.531812 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.531683    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zztwq\" (UniqueName: \"kubernetes.io/projected/76b1753a-5832-4bf4-82e2-7560709c8d80-kube-api-access-zztwq\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.531812 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.531698    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.531812 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.531710    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76b1753a-5832-4bf4-82e2-7560709c8d80-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:04.862600 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.862567    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"]
Jul 06 19:32:04.863053 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.863033    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76b1753a-5832-4bf4-82e2-7560709c8d80" containerName="istio-proxy"
Jul 06 19:32:04.863053 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.863052    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="76b1753a-5832-4bf4-82e2-7560709c8d80" containerName="istio-proxy"
Jul 06 19:32:04.863222 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.863111    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="76b1753a-5832-4bf4-82e2-7560709c8d80" containerName="istio-proxy"
Jul 06 19:32:04.866493 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.866473    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.869640 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.869621    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-d972dd58-openshift-default-dockercfg-gfcqp\""
Jul 06 19:32:04.888197 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.888174    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"]
Jul 06 19:32:04.934877 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.934858    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a54573-cad5-4fb7-8b64-ea052c451287-istiod-ca-cert\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.934973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.934891    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-socket\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.934973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.934919    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-token\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.934973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.934967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-certs\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.935090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.934989    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-envoy\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.935090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.935013    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-credential-socket\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.935090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.935029    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qth5r\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-kube-api-access-qth5r\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.935090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.935045    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-data\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:04.935090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:04.935070    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-podinfo\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036029    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-certs\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036163 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036077    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-envoy\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036216 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036193    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-credential-socket\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036252 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036230    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qth5r\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-kube-api-access-qth5r\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036360 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036338    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-data\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036409 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036386    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-podinfo\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036463 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036450    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a54573-cad5-4fb7-8b64-ea052c451287-istiod-ca-cert\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036515 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036502    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-socket\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036571 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036523    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-certs\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036571 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036560    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-token\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036698 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036573    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-credential-socket\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036778 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036738    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-data\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.036827 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.036808    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-socket\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.037278 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.037255    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a54573-cad5-4fb7-8b64-ea052c451287-istiod-ca-cert\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.038275 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.038252    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-envoy\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.038644 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.038626    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-podinfo\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.048963 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.048935    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-token\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.050918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.050897    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qth5r\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-kube-api-access-qth5r\") pod \"e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") " pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.178792 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.178732    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:05.190807 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.190775    2579 generic.go:356] "Generic (PLEG): container finished" podID="76b1753a-5832-4bf4-82e2-7560709c8d80" containerID="19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b" exitCode=0
Jul 06 19:32:05.190916 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.190809    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96" event={"ID":"76b1753a-5832-4bf4-82e2-7560709c8d80","Type":"ContainerDied","Data":"19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b"}
Jul 06 19:32:05.190916 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.190842    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"
Jul 06 19:32:05.190916 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.190884    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96" event={"ID":"76b1753a-5832-4bf4-82e2-7560709c8d80","Type":"ContainerDied","Data":"7c1c066138aa0950c28845f5e6abaff00cd2bb3eaa343c859d7c5fdf758e776b"}
Jul 06 19:32:05.190916 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.190907    2579 scope.go:117] "RemoveContainer" containerID="19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b"
Jul 06 19:32:05.207560 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.207536    2579 scope.go:117] "RemoveContainer" containerID="19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b"
Jul 06 19:32:05.207903 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:32:05.207874    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b\": container with ID starting with 19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b not found: ID does not exist" containerID="19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b"
Jul 06 19:32:05.207984 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.207913    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b"} err="failed to get container status \"19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b\": rpc error: code = NotFound desc = could not find container \"19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b\": container with ID starting with 19132598937cf7c037c3d890dfcbd0086df31417e0ca74eff12e5083d4a26e7b not found: ID does not exist"
Jul 06 19:32:05.229170 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.229142    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"]
Jul 06 19:32:05.235771 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.235745    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a711c31a-openshift-default-69f6495475-slr96"]
Jul 06 19:32:05.328964 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.328838    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"]
Jul 06 19:32:05.331075 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:32:05.331036    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9a54573_cad5_4fb7_8b64_ea052c451287.slice/crio-eedfba51f645e3f0ab926a5472ad6737723afbc97ccf86d144b8862f896e8367 WatchSource:0}: Error finding container eedfba51f645e3f0ab926a5472ad6737723afbc97ccf86d144b8862f896e8367: Status 404 returned error can't find the container with id eedfba51f645e3f0ab926a5472ad6737723afbc97ccf86d144b8862f896e8367
Jul 06 19:32:05.333243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.333214    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:05.333343 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.333277    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:05.333343 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:05.333310    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:06.198055 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:06.198021    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" event={"ID":"d9a54573-cad5-4fb7-8b64-ea052c451287","Type":"ContainerStarted","Data":"2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3"}
Jul 06 19:32:06.198496 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:06.198082    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" event={"ID":"d9a54573-cad5-4fb7-8b64-ea052c451287","Type":"ContainerStarted","Data":"eedfba51f645e3f0ab926a5472ad6737723afbc97ccf86d144b8862f896e8367"}
Jul 06 19:32:06.262591 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:06.262518    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" podStartSLOduration=2.26250234 podStartE2EDuration="2.26250234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:06.261209247 +0000 UTC m=+1985.872534814" watchObservedRunningTime="2026-07-06 19:32:06.26250234 +0000 UTC m=+1985.873827879"
Jul 06 19:32:06.927998 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:06.927963    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76b1753a-5832-4bf4-82e2-7560709c8d80" path="/var/lib/kubelet/pods/76b1753a-5832-4bf4-82e2-7560709c8d80/volumes"
Jul 06 19:32:07.179285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:07.179204    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:07.183981 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:07.183952    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:07.204271 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:07.204240    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:07.205064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:07.205045    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:17.256887 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:17.256852    2579 generic.go:356] "Generic (PLEG): container finished" podID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerID="2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3" exitCode=0
Jul 06 19:32:17.257282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:17.256905    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" event={"ID":"d9a54573-cad5-4fb7-8b64-ea052c451287","Type":"ContainerDied","Data":"2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3"}
Jul 06 19:32:17.257282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:17.257215    2579 scope.go:117] "RemoveContainer" containerID="2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3"
Jul 06 19:32:17.258150 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:17.258124    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:17.258204 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:17.258192    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:17.258238 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:17.258228    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:18.268470 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:18.268428    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" event={"ID":"d9a54573-cad5-4fb7-8b64-ea052c451287","Type":"ContainerStarted","Data":"c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c"}
Jul 06 19:32:19.178927 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:19.178887    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:19.180023 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:19.179995    2579 patch_prober.go:28] interesting pod/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 06 19:32:19.180152 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:19.180056    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 06 19:32:20.179802 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:20.179762    2579 patch_prober.go:28] interesting pod/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 06 19:32:20.180196 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:20.179818    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 06 19:32:21.179750 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:21.179713    2579 patch_prober.go:28] interesting pod/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 06 19:32:21.179930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:21.179784    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 06 19:32:22.179752 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:22.179718    2579 patch_prober.go:28] interesting pod/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Jul 06 19:32:22.179958 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:22.179774    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Jul 06 19:32:22.205020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:22.204994    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:23.182852 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:23.182826    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:23.183842 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:23.183822    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:33.903548 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:33.903514    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"]
Jul 06 19:32:33.904010 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:33.903744    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy" containerID="cri-o://c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c" gracePeriod=30
Jul 06 19:32:38.185956 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:38.185919    2579 patch_prober.go:28] interesting pod/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 19:32:38.186321 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:38.185978    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 19:32:39.150204 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.150181    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:39.329626 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329600    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-socket\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-data\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329697    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-credential-socket\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329728    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qth5r\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-kube-api-access-qth5r\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-token\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329789    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a54573-cad5-4fb7-8b64-ea052c451287-istiod-ca-cert\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329819    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-envoy\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329844    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-podinfo\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329875    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.329927    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-certs\") pod \"d9a54573-cad5-4fb7-8b64-ea052c451287\" (UID: \"d9a54573-cad5-4fb7-8b64-ea052c451287\") "
Jul 06 19:32:39.330036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.330003    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:39.330577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.330213    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:39.330577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.330264    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.330577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.330283    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.330577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.330441    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9a54573-cad5-4fb7-8b64-ea052c451287-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:32:39.330577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.330471    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-data" (OuterVolumeSpecName: "istio-data") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:39.331974 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.331946    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:39.332103 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.331976    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-kube-api-access-qth5r" (OuterVolumeSpecName: "kube-api-access-qth5r") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "kube-api-access-qth5r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:32:39.332103 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.332085    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-token" (OuterVolumeSpecName: "istio-token") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:32:39.332188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.332120    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d9a54573-cad5-4fb7-8b64-ea052c451287" (UID: "d9a54573-cad5-4fb7-8b64-ea052c451287"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:32:39.378558 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.378536    2579 generic.go:356] "Generic (PLEG): container finished" podID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerID="c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c" exitCode=0
Jul 06 19:32:39.378688 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.378580    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"
Jul 06 19:32:39.378688 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.378595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" event={"ID":"d9a54573-cad5-4fb7-8b64-ea052c451287","Type":"ContainerDied","Data":"c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c"}
Jul 06 19:32:39.378688 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.378638    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8" event={"ID":"d9a54573-cad5-4fb7-8b64-ea052c451287","Type":"ContainerDied","Data":"eedfba51f645e3f0ab926a5472ad6737723afbc97ccf86d144b8862f896e8367"}
Jul 06 19:32:39.378861 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.378686    2579 scope.go:117] "RemoveContainer" containerID="c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c"
Jul 06 19:32:39.387350 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.387243    2579 scope.go:117] "RemoveContainer" containerID="2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3"
Jul 06 19:32:39.394910 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.394894    2579 scope.go:117] "RemoveContainer" containerID="c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c"
Jul 06 19:32:39.395155 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:32:39.395137    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c\": container with ID starting with c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c not found: ID does not exist" containerID="c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c"
Jul 06 19:32:39.395207 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.395165    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c"} err="failed to get container status \"c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c\": rpc error: code = NotFound desc = could not find container \"c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c\": container with ID starting with c394a5a2bc69a2d8759cec20592b43cdd3c2696d0ab3eba2d7d9ddc9faa60b3c not found: ID does not exist"
Jul 06 19:32:39.395207 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.395184    2579 scope.go:117] "RemoveContainer" containerID="2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3"
Jul 06 19:32:39.395404 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:32:39.395387    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3\": container with ID starting with 2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3 not found: ID does not exist" containerID="2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3"
Jul 06 19:32:39.395455 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.395411    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3"} err="failed to get container status \"2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3\": rpc error: code = NotFound desc = could not find container \"2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3\": container with ID starting with 2ed6f8ac92929c63e3238ace743593144102a183f1a26a4eb619f9902fe560b3 not found: ID does not exist"
Jul 06 19:32:39.402155 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.402135    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"]
Jul 06 19:32:39.412395 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.412375    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-d972dd58-openshift-default-667fbcd8cc-6z7f8"]
Jul 06 19:32:39.431468 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.431450    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.431565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.431471    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.431565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.431484    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qth5r\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-kube-api-access-qth5r\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.431565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.431498    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.431565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.431510    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d9a54573-cad5-4fb7-8b64-ea052c451287-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.431565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.431522    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:39.431565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:39.431535    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d9a54573-cad5-4fb7-8b64-ea052c451287-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:40.927579 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:40.927545    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" path="/var/lib/kubelet/pods/d9a54573-cad5-4fb7-8b64-ea052c451287/volumes"
Jul 06 19:32:43.420502 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.420471    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"]
Jul 06 19:32:43.420890 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.420811    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy"
Jul 06 19:32:43.420890 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.420822    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy"
Jul 06 19:32:43.420890 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.420884    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy"
Jul 06 19:32:43.420890 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.420892    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy"
Jul 06 19:32:43.425379 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.425358    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.428370 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.428351    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-ec6a8779-openshift-default-dockercfg-r9l86\""
Jul 06 19:32:43.430242 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.430221    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 19:32:43.450043 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.450000    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"]
Jul 06 19:32:43.465617 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465590    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-credential-socket\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.465751 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465633    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/698df663-593d-408d-9451-ce99b7fd6639-istio-podinfo\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.465751 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465732    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/698df663-593d-408d-9451-ce99b7fd6639-istiod-ca-cert\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.465838 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465821    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-socket\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.465883 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-envoy\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.465937 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465890    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-istio-token\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.465990 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465941    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-data\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.465990 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.465971    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-certs\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.466074 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.466007    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wp7t7\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-kube-api-access-wp7t7\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.566820 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.566779    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/698df663-593d-408d-9451-ce99b7fd6639-istio-podinfo\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567021 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.566857    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/698df663-593d-408d-9451-ce99b7fd6639-istiod-ca-cert\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567021 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.566909    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-socket\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567021 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.566930    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-envoy\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567021 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.566955    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-istio-token\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567021 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.566981    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-data\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567021 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567014    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-certs\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567339 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567050    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wp7t7\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-kube-api-access-wp7t7\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567339 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567080    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-credential-socket\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567458 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567386    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-socket\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567600 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567569    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-certs\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567761 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567608    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-credential-socket\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567761 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567671    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-data\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.567761 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.567728    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/698df663-593d-408d-9451-ce99b7fd6639-istiod-ca-cert\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.569331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.569311    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-envoy\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.569783 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.569768    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/698df663-593d-408d-9451-ce99b7fd6639-istio-podinfo\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.577255 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.577231    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-istio-token\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.578791 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.578774    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wp7t7\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-kube-api-access-wp7t7\") pod \"e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") " pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.735428 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.735343    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:43.922566 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.922536    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"]
Jul 06 19:32:43.925904 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:32:43.925868    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod698df663_593d_408d_9451_ce99b7fd6639.slice/crio-ea78976d7acfe40aaa534475aeb8a35a61d5182b8704eff4bb1b102a3b842a4d WatchSource:0}: Error finding container ea78976d7acfe40aaa534475aeb8a35a61d5182b8704eff4bb1b102a3b842a4d: Status 404 returned error can't find the container with id ea78976d7acfe40aaa534475aeb8a35a61d5182b8704eff4bb1b102a3b842a4d
Jul 06 19:32:43.927880 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.927852    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:43.927956 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.927911    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:43.927956 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:43.927936    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:32:44.405904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:44.405875    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs" event={"ID":"698df663-593d-408d-9451-ce99b7fd6639","Type":"ContainerStarted","Data":"5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f"}
Jul 06 19:32:44.406084 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:44.405922    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs" event={"ID":"698df663-593d-408d-9451-ce99b7fd6639","Type":"ContainerStarted","Data":"ea78976d7acfe40aaa534475aeb8a35a61d5182b8704eff4bb1b102a3b842a4d"}
Jul 06 19:32:44.450005 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:44.449948    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs" podStartSLOduration=1.449928152 podStartE2EDuration="1.449928152s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:32:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:32:44.446971617 +0000 UTC m=+2024.058297156" watchObservedRunningTime="2026-07-06 19:32:44.449928152 +0000 UTC m=+2024.061253694"
Jul 06 19:32:44.735965 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:44.735886    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:44.740788 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:44.740763    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:45.410950 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:45.410921    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:45.411966 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:45.411948    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:47.004413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:47.004376    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"]
Jul 06 19:32:47.420815 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:47.420784    2579 kubelet_pods.go:1082] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs" secret="" err="secret \"e2e-derive-ec6a8779-openshift-default-dockercfg-r9l86\" not found"
Jul 06 19:32:48.426133 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:48.426077    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs" podUID="698df663-593d-408d-9451-ce99b7fd6639" containerName="istio-proxy" containerID="cri-o://5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f" gracePeriod=30
Jul 06 19:32:54.164049 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.163991    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:54.256350 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256285    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-envoy\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256350 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256327    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/698df663-593d-408d-9451-ce99b7fd6639-istio-podinfo\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256381    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-data\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256423    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-socket\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256445    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-certs\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256480    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-istio-token\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256512    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-credential-socket\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256567    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/698df663-593d-408d-9451-ce99b7fd6639-istiod-ca-cert\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256612    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wp7t7\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-kube-api-access-wp7t7\") pod \"698df663-593d-408d-9451-ce99b7fd6639\" (UID: \"698df663-593d-408d-9451-ce99b7fd6639\") "
Jul 06 19:32:54.256813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256746    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:54.256979 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256925    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:54.256979 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.256940    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.257162 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.257138    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:54.257232 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.257163    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-data" (OuterVolumeSpecName: "istio-data") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:54.257313 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.257294    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/698df663-593d-408d-9451-ce99b7fd6639-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:32:54.258478 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.258457    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/698df663-593d-408d-9451-ce99b7fd6639-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:32:54.258763 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.258748    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-kube-api-access-wp7t7" (OuterVolumeSpecName: "kube-api-access-wp7t7") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "kube-api-access-wp7t7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:32:54.258878 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.258858    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-istio-token" (OuterVolumeSpecName: "istio-token") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:32:54.258878 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.258864    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "698df663-593d-408d-9451-ce99b7fd6639" (UID: "698df663-593d-408d-9451-ce99b7fd6639"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:32:54.357947 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.357916    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wp7t7\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-kube-api-access-wp7t7\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.357947 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.357948    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.358159 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.357961    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/698df663-593d-408d-9451-ce99b7fd6639-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.358159 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.357972    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.358159 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.357984    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.358159 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.357995    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/698df663-593d-408d-9451-ce99b7fd6639-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.358159 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.358005    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/698df663-593d-408d-9451-ce99b7fd6639-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.358159 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.358017    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/698df663-593d-408d-9451-ce99b7fd6639-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:32:54.461010 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.460968    2579 generic.go:356] "Generic (PLEG): container finished" podID="698df663-593d-408d-9451-ce99b7fd6639" containerID="5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f" exitCode=0
Jul 06 19:32:54.461153 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.461028    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"
Jul 06 19:32:54.461153 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.461061    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs" event={"ID":"698df663-593d-408d-9451-ce99b7fd6639","Type":"ContainerDied","Data":"5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f"}
Jul 06 19:32:54.461153 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.461139    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs" event={"ID":"698df663-593d-408d-9451-ce99b7fd6639","Type":"ContainerDied","Data":"ea78976d7acfe40aaa534475aeb8a35a61d5182b8704eff4bb1b102a3b842a4d"}
Jul 06 19:32:54.461296 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.461162    2579 scope.go:117] "RemoveContainer" containerID="5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f"
Jul 06 19:32:54.470561 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.470529    2579 scope.go:117] "RemoveContainer" containerID="5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f"
Jul 06 19:32:54.470800 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:32:54.470777    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f\": container with ID starting with 5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f not found: ID does not exist" containerID="5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f"
Jul 06 19:32:54.470895 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.470803    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f"} err="failed to get container status \"5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f\": rpc error: code = NotFound desc = could not find container \"5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f\": container with ID starting with 5d7f90422e5b4947be4e2a547413b0a027d41d3f1504560e2da2bf529d81aa2f not found: ID does not exist"
Jul 06 19:32:54.495163 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.495141    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"]
Jul 06 19:32:54.504355 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.504333    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-ec6a8779-openshift-default-75c8c44c7f-94zrs"]
Jul 06 19:32:54.927254 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:32:54.927223    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="698df663-593d-408d-9451-ce99b7fd6639" path="/var/lib/kubelet/pods/698df663-593d-408d-9451-ce99b7fd6639/volumes"
Jul 06 19:34:42.862632 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.862598    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"]
Jul 06 19:34:42.863154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.863081    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy"
Jul 06 19:34:42.863154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.863096    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9a54573-cad5-4fb7-8b64-ea052c451287" containerName="istio-proxy"
Jul 06 19:34:42.863154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.863134    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="698df663-593d-408d-9451-ce99b7fd6639" containerName="istio-proxy"
Jul 06 19:34:42.863154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.863142    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="698df663-593d-408d-9451-ce99b7fd6639" containerName="istio-proxy"
Jul 06 19:34:42.863291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.863214    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="698df663-593d-408d-9451-ce99b7fd6639" containerName="istio-proxy"
Jul 06 19:34:42.866409 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.866393    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.869645 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.869616    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 19:34:42.891555 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.891524    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"]
Jul 06 19:34:42.998939 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.998901    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-socket\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.998939 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.998937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-podinfo\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.999176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.998965    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-data\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.999176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.999044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-certs\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.999176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.999116    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-credential-socket\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.999176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.999160    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb9nq\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-kube-api-access-kb9nq\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.999362 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.999206    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-envoy\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.999362 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.999229    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-token\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:42.999362 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:42.999275    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istiod-ca-cert\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.100634 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.100604    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-credential-socket\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.100840 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.100646    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb9nq\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-kube-api-access-kb9nq\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.100840 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.100778    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-envoy\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.100840 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.100813    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-token\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.100840 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.100841    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istiod-ca-cert\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.100911    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-socket\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.100931    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-podinfo\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.101047    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-data\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.101062    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-credential-socket\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101289 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.101089    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-certs\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101412 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.101389    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-socket\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101486 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.101397    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-data\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101486 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.101433    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-certs\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.101709 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.101686    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istiod-ca-cert\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.103273 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.103253    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-podinfo\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.103348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.103313    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-envoy\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.112468 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.112449    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb9nq\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-kube-api-access-kb9nq\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.113104 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.113064    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-token\") pod \"e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") " pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.176378 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.176343    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:43.340530 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.340497    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"]
Jul 06 19:34:43.343520 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:34:43.343493    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2c36f4_4c8a_470f_a7d8_bc84217b05db.slice/crio-e13382af9fc12fa7b638d0d3c5822178b6e79d66f9c306e79545904eed6a4174 WatchSource:0}: Error finding container e13382af9fc12fa7b638d0d3c5822178b6e79d66f9c306e79545904eed6a4174: Status 404 returned error can't find the container with id e13382af9fc12fa7b638d0d3c5822178b6e79d66f9c306e79545904eed6a4174
Jul 06 19:34:43.345735 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.345696    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:34:43.345811 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.345787    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:34:43.345855 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:43.345827    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:34:44.000909 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:44.000876    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" event={"ID":"0d2c36f4-4c8a-470f-a7d8-bc84217b05db","Type":"ContainerStarted","Data":"c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8"}
Jul 06 19:34:44.001294 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:44.000925    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" event={"ID":"0d2c36f4-4c8a-470f-a7d8-bc84217b05db","Type":"ContainerStarted","Data":"e13382af9fc12fa7b638d0d3c5822178b6e79d66f9c306e79545904eed6a4174"}
Jul 06 19:34:44.177258 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:44.177228    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:44.182312 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:44.182288    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:44.249973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:44.249926    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" podStartSLOduration=2.249908999 podStartE2EDuration="2.249908999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:34:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:34:44.034353347 +0000 UTC m=+2143.645678887" watchObservedRunningTime="2026-07-06 19:34:44.249908999 +0000 UTC m=+2143.861234537"
Jul 06 19:34:45.011013 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:45.007783    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:34:45.011988 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:34:45.011960    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:35:00.881266 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:00.881235    2579 patch_prober.go:28] interesting pod/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:47236->10.133.0.48:15021: read: connection reset by peer" start-of-body=
Jul 06 19:35:00.881704 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:00.881312    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": read tcp 10.133.0.2:47236->10.133.0.48:15021: read: connection reset by peer"
Jul 06 19:35:01.090650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:01.090582    2579 generic.go:356] "Generic (PLEG): container finished" podID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerID="c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8" exitCode=0
Jul 06 19:35:01.090791 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:01.090676    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" event={"ID":"0d2c36f4-4c8a-470f-a7d8-bc84217b05db","Type":"ContainerDied","Data":"c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8"}
Jul 06 19:35:01.091101 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:01.091084    2579 scope.go:117] "RemoveContainer" containerID="c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8"
Jul 06 19:35:01.092060 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:01.092033    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:01.092140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:01.092114    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:01.092177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:01.092169    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:02.099204 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:02.099168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" event={"ID":"0d2c36f4-4c8a-470f-a7d8-bc84217b05db","Type":"ContainerStarted","Data":"efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869"}
Jul 06 19:35:02.177390 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:02.177351    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:35:02.177586 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:02.177563    2579 patch_prober.go:28] interesting pod/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 06 19:35:02.177639 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:02.177618    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 06 19:35:03.176868 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:03.176829    2579 patch_prober.go:28] interesting pod/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 06 19:35:03.177240 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:03.176883    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 06 19:35:04.176960 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:04.176926    2579 patch_prober.go:28] interesting pod/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 06 19:35:04.177332 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:04.176980    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 06 19:35:05.176717 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:05.176680    2579 patch_prober.go:28] interesting pod/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body=
Jul 06 19:35:05.176882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:05.176737    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused"
Jul 06 19:35:05.773046 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:05.773012    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"]
Jul 06 19:35:05.773430 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:05.773209    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy" containerID="cri-o://efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869" gracePeriod=30
Jul 06 19:35:11.523418 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.523393    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:35:11.642885 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.642814    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-token\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.642885 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.642845    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-socket\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.642885 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.642875    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-credential-socket\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.643143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.642891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb9nq\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-kube-api-access-kb9nq\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.643143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643011    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istiod-ca-cert\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.643143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643048    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-certs\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.643143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643070    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-envoy\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.643143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643108    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-podinfo\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.643423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643193    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-data\") pod \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\" (UID: \"0d2c36f4-4c8a-470f-a7d8-bc84217b05db\") "
Jul 06 19:35:11.643483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643186    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:35:11.643483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643332    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:35:11.643483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643418    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:35:11.643649 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.643510    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:35:11.644213 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.644182    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-data" (OuterVolumeSpecName: "istio-data") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:35:11.645235 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.645212    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-token" (OuterVolumeSpecName: "istio-token") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:35:11.645302 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.645216    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:35:11.645302 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.645257    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-kube-api-access-kb9nq" (OuterVolumeSpecName: "kube-api-access-kb9nq") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "kube-api-access-kb9nq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:35:11.645302 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.645256    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0d2c36f4-4c8a-470f-a7d8-bc84217b05db" (UID: "0d2c36f4-4c8a-470f-a7d8-bc84217b05db"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:35:11.743996 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.743974    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.743996 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.743995    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.744122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.744004    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.744122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.744013    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.744122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.744029    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb9nq\" (UniqueName: \"kubernetes.io/projected/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-kube-api-access-kb9nq\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.744122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.744038    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.744122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.744046    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.744122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.744053    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:11.744122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:11.744060    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0d2c36f4-4c8a-470f-a7d8-bc84217b05db-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:35:12.154519 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.154205    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" event={"ID":"0d2c36f4-4c8a-470f-a7d8-bc84217b05db","Type":"ContainerDied","Data":"efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869"}
Jul 06 19:35:12.154519 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.154322    2579 scope.go:117] "RemoveContainer" containerID="efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869"
Jul 06 19:35:12.154789 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.154556    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"
Jul 06 19:35:12.154789 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.154547    2579 generic.go:356] "Generic (PLEG): container finished" podID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerID="efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869" exitCode=0
Jul 06 19:35:12.154896 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.154775    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk" event={"ID":"0d2c36f4-4c8a-470f-a7d8-bc84217b05db","Type":"ContainerDied","Data":"e13382af9fc12fa7b638d0d3c5822178b6e79d66f9c306e79545904eed6a4174"}
Jul 06 19:35:12.168384 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.168362    2579 scope.go:117] "RemoveContainer" containerID="c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8"
Jul 06 19:35:12.176020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.176004    2579 scope.go:117] "RemoveContainer" containerID="efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869"
Jul 06 19:35:12.176247 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:35:12.176229    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869\": container with ID starting with efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869 not found: ID does not exist" containerID="efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869"
Jul 06 19:35:12.176304 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.176255    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869"} err="failed to get container status \"efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869\": rpc error: code = NotFound desc = could not find container \"efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869\": container with ID starting with efa5b37581e26aa1d927668f955588070a90d89134882c5f6e58eb23e824d869 not found: ID does not exist"
Jul 06 19:35:12.176304 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.176271    2579 scope.go:117] "RemoveContainer" containerID="c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8"
Jul 06 19:35:12.176493 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:35:12.176478    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8\": container with ID starting with c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8 not found: ID does not exist" containerID="c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8"
Jul 06 19:35:12.176535 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.176496    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8"} err="failed to get container status \"c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8\": rpc error: code = NotFound desc = could not find container \"c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8\": container with ID starting with c67194d1d6e9e84e21c5b5e5f1e7ead068f986a60a97a534a99c88eed46836c8 not found: ID does not exist"
Jul 06 19:35:12.182372 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.182352    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"]
Jul 06 19:35:12.189382 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.189360    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-95f72a38-openshift-default-6bb54d89f9-xktvk"]
Jul 06 19:35:12.927510 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:12.927481    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" path="/var/lib/kubelet/pods/0d2c36f4-4c8a-470f-a7d8-bc84217b05db/volumes"
Jul 06 19:35:27.075728 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.075693    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"]
Jul 06 19:35:27.076177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.076035    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy"
Jul 06 19:35:27.076177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.076047    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy"
Jul 06 19:35:27.076177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.076116    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy"
Jul 06 19:35:27.076177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.076124    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy"
Jul 06 19:35:27.080285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.080266    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.085243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.085223    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3ed9b9ce-openshift-default-dockercfg-nqbbr\""
Jul 06 19:35:27.086382 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.086363    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 19:35:27.099422 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.099398    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"]
Jul 06 19:35:27.174160 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174124    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6f2s\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-kube-api-access-b6f2s\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174327 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-token\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174327 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174191    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-envoy\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174327 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174251    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-socket\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174327 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174273    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-podinfo\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174327 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174292    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-credential-socket\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174526 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174329    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfc107-c8cc-453e-8488-7851e1e9356c-istiod-ca-cert\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174526 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174439    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-certs\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.174526 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.174480    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-data\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275301 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275268    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6f2s\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-kube-api-access-b6f2s\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275301 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275306    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-token\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275538 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275333    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-envoy\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275538 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275512    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-socket\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275711 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275556    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-podinfo\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275711 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275593    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-credential-socket\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275711 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275697    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfc107-c8cc-453e-8488-7851e1e9356c-istiod-ca-cert\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275884 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275777    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-certs\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275884 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275803    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-data\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.275996 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275896    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-socket\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.276096 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.275986    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-credential-socket\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.276194 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.276174    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-data\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.276259 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.276212    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-certs\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.276588 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.276558    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfc107-c8cc-453e-8488-7851e1e9356c-istiod-ca-cert\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.277858 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.277832    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-envoy\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.278380 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.278358    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-podinfo\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.287585 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.287563    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-token\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.288882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.288865    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6f2s\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-kube-api-access-b6f2s\") pod \"e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") " pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.391990 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.391900    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:27.559370 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.559344    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"]
Jul 06 19:35:27.560248 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:35:27.560225    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbbfc107_c8cc_453e_8488_7851e1e9356c.slice/crio-60b7c2a93f42aebb84bdd23bbfd4ed6755436a41489e1cf31ce6ec263af34bda WatchSource:0}: Error finding container 60b7c2a93f42aebb84bdd23bbfd4ed6755436a41489e1cf31ce6ec263af34bda: Status 404 returned error can't find the container with id 60b7c2a93f42aebb84bdd23bbfd4ed6755436a41489e1cf31ce6ec263af34bda
Jul 06 19:35:27.562253 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.562224    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:27.562315 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.562284    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:27.562315 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:27.562312    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:28.238736 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:28.238696    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" event={"ID":"cbbfc107-c8cc-453e-8488-7851e1e9356c","Type":"ContainerStarted","Data":"4243f21b1d7f819ef2cb6f35b7ed917bace0f3491c0ad5297e0cee2e6f4695db"}
Jul 06 19:35:28.239111 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:28.238761    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" event={"ID":"cbbfc107-c8cc-453e-8488-7851e1e9356c","Type":"ContainerStarted","Data":"60b7c2a93f42aebb84bdd23bbfd4ed6755436a41489e1cf31ce6ec263af34bda"}
Jul 06 19:35:28.268317 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:28.268273    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" podStartSLOduration=1.268257722 podStartE2EDuration="1.268257722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:28.267036487 +0000 UTC m=+2187.878362025" watchObservedRunningTime="2026-07-06 19:35:28.268257722 +0000 UTC m=+2187.879583329"
Jul 06 19:35:28.392298 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:28.392271    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:28.397134 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:28.397110    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:29.243600 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:29.243575    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:29.244333 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:29.244317    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:43.315686 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:43.315637    2579 generic.go:356] "Generic (PLEG): container finished" podID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerID="4243f21b1d7f819ef2cb6f35b7ed917bace0f3491c0ad5297e0cee2e6f4695db" exitCode=0
Jul 06 19:35:43.316120 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:43.315692    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" event={"ID":"cbbfc107-c8cc-453e-8488-7851e1e9356c","Type":"ContainerDied","Data":"4243f21b1d7f819ef2cb6f35b7ed917bace0f3491c0ad5297e0cee2e6f4695db"}
Jul 06 19:35:43.316120 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:43.316039    2579 scope.go:117] "RemoveContainer" containerID="4243f21b1d7f819ef2cb6f35b7ed917bace0f3491c0ad5297e0cee2e6f4695db"
Jul 06 19:35:43.316997 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:43.316968    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:43.317072 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:43.317056    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:43.317118 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:43.317103    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:35:44.244390 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:44.244346    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:44.323036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:44.323006    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" event={"ID":"cbbfc107-c8cc-453e-8488-7851e1e9356c","Type":"ContainerStarted","Data":"f6b237bc6a4fd2de8e714c8c1b7b4d28f936e816111f91eb777c875dda11d44e"}
Jul 06 19:35:44.392291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:44.392255    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:44.393191 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:44.393169    2579 patch_prober.go:28] interesting pod/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 06 19:35:44.393326 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:44.393221    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 06 19:35:45.392599 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:45.392563    2579 patch_prober.go:28] interesting pod/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 06 19:35:45.392993 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:45.392614    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 06 19:35:46.393151 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:46.393117    2579 patch_prober.go:28] interesting pod/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 06 19:35:46.393613 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:46.393208    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 06 19:35:47.392640 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:47.392604    2579 patch_prober.go:28] interesting pod/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused" start-of-body=
Jul 06 19:35:47.392844 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:47.392691    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.49:15021/healthz/ready\": dial tcp 10.133.0.49:15021: connect: connection refused"
Jul 06 19:35:48.435163 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:48.435130    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:48.435566 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:48.435461    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:48.436127 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:48.436108    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:35:54.985080 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.985048    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"]
Jul 06 19:35:54.985446 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.985396    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy"
Jul 06 19:35:54.985446 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.985408    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="0d2c36f4-4c8a-470f-a7d8-bc84217b05db" containerName="istio-proxy"
Jul 06 19:35:54.988233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.988217    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:54.990927 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.990905    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3ed9b9ce\"/\"e2e-lifecycle-model-3ed9b9ce-kserve-self-signed-certs\""
Jul 06 19:35:54.992337 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.992310    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3ed9b9ce\"/\"kube-root-ca.crt\""
Jul 06 19:35:54.992444 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.992394    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-3ed9b9ce\"/\"default-dockercfg-bj62f\""
Jul 06 19:35:54.993071 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:54.993058    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-3ed9b9ce\"/\"openshift-service-ca.crt\""
Jul 06 19:35:55.000525 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.000505    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"]
Jul 06 19:35:55.104530 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.104499    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzf9v\" (UniqueName: \"kubernetes.io/projected/a158e7ca-9e93-4339-b291-59b594ad4a4e-kube-api-access-pzf9v\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.104692 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.104535    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-dshm\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.104692 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.104562    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-home\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.104692 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.104617    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-model-cache\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.104692 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.104679    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a158e7ca-9e93-4339-b291-59b594ad4a4e-tls-certs\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.104837 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.104718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-tmp-dir\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.205543 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.205508    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-model-cache\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.205699 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.205555    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a158e7ca-9e93-4339-b291-59b594ad4a4e-tls-certs\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.205699 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.205591    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-tmp-dir\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.205699 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.205618    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzf9v\" (UniqueName: \"kubernetes.io/projected/a158e7ca-9e93-4339-b291-59b594ad4a4e-kube-api-access-pzf9v\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.205699 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.205673    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-dshm\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.205924 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.205713    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-home\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.205984 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.205957    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-model-cache\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.206040 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.206013    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-tmp-dir\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.206083 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.206063    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-home\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.207814 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.207791    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-dshm\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.208125 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.208110    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a158e7ca-9e93-4339-b291-59b594ad4a4e-tls-certs\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.217224 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.217203    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzf9v\" (UniqueName: \"kubernetes.io/projected/a158e7ca-9e93-4339-b291-59b594ad4a4e-kube-api-access-pzf9v\") pod \"e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") " pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.297958 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.297935    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:35:55.444411 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:55.444376    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"]
Jul 06 19:35:55.448114 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:35:55.448081    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda158e7ca_9e93_4339_b291_59b594ad4a4e.slice/crio-db03b898e2dbd9c5eed01098a874b22cbc7f1207e6bd6f8922fea0a6fb48278d WatchSource:0}: Error finding container db03b898e2dbd9c5eed01098a874b22cbc7f1207e6bd6f8922fea0a6fb48278d: Status 404 returned error can't find the container with id db03b898e2dbd9c5eed01098a874b22cbc7f1207e6bd6f8922fea0a6fb48278d
Jul 06 19:35:56.389377 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:56.389341    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg" event={"ID":"a158e7ca-9e93-4339-b291-59b594ad4a4e","Type":"ContainerStarted","Data":"f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06"}
Jul 06 19:35:56.389765 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:56.389382    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg" event={"ID":"a158e7ca-9e93-4339-b291-59b594ad4a4e","Type":"ContainerStarted","Data":"db03b898e2dbd9c5eed01098a874b22cbc7f1207e6bd6f8922fea0a6fb48278d"}
Jul 06 19:35:56.412958 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:35:56.412913    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg" podStartSLOduration=2.412898428 podStartE2EDuration="2.412898428s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:35:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:35:56.410140864 +0000 UTC m=+2216.021466402" watchObservedRunningTime="2026-07-06 19:35:56.412898428 +0000 UTC m=+2216.024223966"
Jul 06 19:36:05.298705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:05.298650    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:36:05.299219 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:05.298722    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:36:05.311043 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:05.311016    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:36:05.445487 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:05.445458    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:36:28.594463 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:28.594384    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"]
Jul 06 19:36:43.614453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.614408    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg" podUID="a158e7ca-9e93-4339-b291-59b594ad4a4e" containerName="main" containerID="cri-o://f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06" gracePeriod=45
Jul 06 19:36:43.855519 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.855498    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:36:43.886681 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886622    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-model-cache\") pod \"a158e7ca-9e93-4339-b291-59b594ad4a4e\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") "
Jul 06 19:36:43.886773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886687    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-home\") pod \"a158e7ca-9e93-4339-b291-59b594ad4a4e\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") "
Jul 06 19:36:43.886773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886720    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-tmp-dir\") pod \"a158e7ca-9e93-4339-b291-59b594ad4a4e\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") "
Jul 06 19:36:43.886773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886760    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzf9v\" (UniqueName: \"kubernetes.io/projected/a158e7ca-9e93-4339-b291-59b594ad4a4e-kube-api-access-pzf9v\") pod \"a158e7ca-9e93-4339-b291-59b594ad4a4e\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") "
Jul 06 19:36:43.886944 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886785    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-dshm\") pod \"a158e7ca-9e93-4339-b291-59b594ad4a4e\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") "
Jul 06 19:36:43.886944 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886865    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a158e7ca-9e93-4339-b291-59b594ad4a4e-tls-certs\") pod \"a158e7ca-9e93-4339-b291-59b594ad4a4e\" (UID: \"a158e7ca-9e93-4339-b291-59b594ad4a4e\") "
Jul 06 19:36:43.886944 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886899    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-model-cache" (OuterVolumeSpecName: "model-cache") pod "a158e7ca-9e93-4339-b291-59b594ad4a4e" (UID: "a158e7ca-9e93-4339-b291-59b594ad4a4e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:36:43.887093 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886974    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-home" (OuterVolumeSpecName: "home") pod "a158e7ca-9e93-4339-b291-59b594ad4a4e" (UID: "a158e7ca-9e93-4339-b291-59b594ad4a4e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:36:43.887093 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.886988    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a158e7ca-9e93-4339-b291-59b594ad4a4e" (UID: "a158e7ca-9e93-4339-b291-59b594ad4a4e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:36:43.887195 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.887112    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:36:43.887195 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.887129    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:36:43.887195 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.887142    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:36:43.888941 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.888916    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a158e7ca-9e93-4339-b291-59b594ad4a4e-kube-api-access-pzf9v" (OuterVolumeSpecName: "kube-api-access-pzf9v") pod "a158e7ca-9e93-4339-b291-59b594ad4a4e" (UID: "a158e7ca-9e93-4339-b291-59b594ad4a4e"). InnerVolumeSpecName "kube-api-access-pzf9v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:36:43.889043 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.888980    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a158e7ca-9e93-4339-b291-59b594ad4a4e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a158e7ca-9e93-4339-b291-59b594ad4a4e" (UID: "a158e7ca-9e93-4339-b291-59b594ad4a4e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:36:43.889043 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.888992    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-dshm" (OuterVolumeSpecName: "dshm") pod "a158e7ca-9e93-4339-b291-59b594ad4a4e" (UID: "a158e7ca-9e93-4339-b291-59b594ad4a4e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:36:43.987770 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.987736    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzf9v\" (UniqueName: \"kubernetes.io/projected/a158e7ca-9e93-4339-b291-59b594ad4a4e-kube-api-access-pzf9v\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:36:43.987770 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.987765    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a158e7ca-9e93-4339-b291-59b594ad4a4e-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:36:43.987770 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:43.987777    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a158e7ca-9e93-4339-b291-59b594ad4a4e-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:36:44.628241 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.628206    2579 generic.go:356] "Generic (PLEG): container finished" podID="a158e7ca-9e93-4339-b291-59b594ad4a4e" containerID="f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06" exitCode=0
Jul 06 19:36:44.628650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.628251    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"
Jul 06 19:36:44.628650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.628291    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg" event={"ID":"a158e7ca-9e93-4339-b291-59b594ad4a4e","Type":"ContainerDied","Data":"f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06"}
Jul 06 19:36:44.628650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.628336    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg" event={"ID":"a158e7ca-9e93-4339-b291-59b594ad4a4e","Type":"ContainerDied","Data":"db03b898e2dbd9c5eed01098a874b22cbc7f1207e6bd6f8922fea0a6fb48278d"}
Jul 06 19:36:44.628650 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.628355    2579 scope.go:117] "RemoveContainer" containerID="f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06"
Jul 06 19:36:44.637465 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.637450    2579 scope.go:117] "RemoveContainer" containerID="f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06"
Jul 06 19:36:44.637731 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:36:44.637710    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06\": container with ID starting with f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06 not found: ID does not exist" containerID="f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06"
Jul 06 19:36:44.637799 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.637742    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06"} err="failed to get container status \"f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06\": rpc error: code = NotFound desc = could not find container \"f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06\": container with ID starting with f2e4b50af5553722bfd24909e29b7893e5c9e3e93e40d99d2d8d4c86bd57ec06 not found: ID does not exist"
Jul 06 19:36:44.651964 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.651943    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"]
Jul 06 19:36:44.653943 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.653925    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-3ed9b9ce/e2e-lifecycle-model-3ed9b9ce-kserve-5c46d795-6c7sg"]
Jul 06 19:36:44.927064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:36:44.926996    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a158e7ca-9e93-4339-b291-59b594ad4a4e" path="/var/lib/kubelet/pods/a158e7ca-9e93-4339-b291-59b594ad4a4e/volumes"
Jul 06 19:37:32.767426 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:32.767392    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"]
Jul 06 19:37:32.767917 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:32.767601    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy" containerID="cri-o://f6b237bc6a4fd2de8e714c8c1b7b4d28f936e816111f91eb777c875dda11d44e" gracePeriod=30
Jul 06 19:37:33.437759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:33.437722    2579 patch_prober.go:28] interesting pod/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 19:37:33.437913 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:33.437775    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 19:37:37.893025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:37.892993    2579 generic.go:356] "Generic (PLEG): container finished" podID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerID="f6b237bc6a4fd2de8e714c8c1b7b4d28f936e816111f91eb777c875dda11d44e" exitCode=0
Jul 06 19:37:37.893346 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:37.893060    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" event={"ID":"cbbfc107-c8cc-453e-8488-7851e1e9356c","Type":"ContainerDied","Data":"f6b237bc6a4fd2de8e714c8c1b7b4d28f936e816111f91eb777c875dda11d44e"}
Jul 06 19:37:37.893346 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:37.893120    2579 scope.go:117] "RemoveContainer" containerID="4243f21b1d7f819ef2cb6f35b7ed917bace0f3491c0ad5297e0cee2e6f4695db"
Jul 06 19:37:38.029484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.029460    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:37:38.117022 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.116989    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfc107-c8cc-453e-8488-7851e1e9356c-istiod-ca-cert\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117192 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117077    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-certs\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117192 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117110    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-token\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117192 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117153    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-envoy\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117192 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117181    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6f2s\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-kube-api-access-b6f2s\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117210    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-podinfo\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-credential-socket\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117294    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-socket\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117342    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-data\") pod \"cbbfc107-c8cc-453e-8488-7851e1e9356c\" (UID: \"cbbfc107-c8cc-453e-8488-7851e1e9356c\") "
Jul 06 19:37:38.117423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117372    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbbfc107-c8cc-453e-8488-7851e1e9356c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:37:38.117423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117400    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:37:38.117762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117560    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:37:38.117762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117633    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.117762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117651    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cbbfc107-c8cc-453e-8488-7851e1e9356c-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.117762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117693    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.117967 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.117757    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:37:38.118183 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.118163    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-data" (OuterVolumeSpecName: "istio-data") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:37:38.119406 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.119386    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-token" (OuterVolumeSpecName: "istio-token") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:37:38.119507 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.119412    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-kube-api-access-b6f2s" (OuterVolumeSpecName: "kube-api-access-b6f2s") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "kube-api-access-b6f2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:37:38.119745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.119720    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:37:38.119851 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.119824    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cbbfc107-c8cc-453e-8488-7851e1e9356c" (UID: "cbbfc107-c8cc-453e-8488-7851e1e9356c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:37:38.218188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.218154    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.218188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.218183    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.218188 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.218193    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6f2s\" (UniqueName: \"kubernetes.io/projected/cbbfc107-c8cc-453e-8488-7851e1e9356c-kube-api-access-b6f2s\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.218413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.218202    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.218413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.218211    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.218413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.218219    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cbbfc107-c8cc-453e-8488-7851e1e9356c-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:37:38.899462 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.899433    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"
Jul 06 19:37:38.899904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.899471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72" event={"ID":"cbbfc107-c8cc-453e-8488-7851e1e9356c","Type":"ContainerDied","Data":"60b7c2a93f42aebb84bdd23bbfd4ed6755436a41489e1cf31ce6ec263af34bda"}
Jul 06 19:37:38.899904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.899522    2579 scope.go:117] "RemoveContainer" containerID="f6b237bc6a4fd2de8e714c8c1b7b4d28f936e816111f91eb777c875dda11d44e"
Jul 06 19:37:38.939262 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.939220    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"]
Jul 06 19:37:38.945174 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:38.945152    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3ed9b9ce-openshift-default-78c9cdcb87-nnq72"]
Jul 06 19:37:40.927168 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:37:40.927137    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" path="/var/lib/kubelet/pods/cbbfc107-c8cc-453e-8488-7851e1e9356c/volumes"
Jul 06 19:38:29.798078 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798042    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"]
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798348    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798358    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798369    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798377    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798385    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a158e7ca-9e93-4339-b291-59b594ad4a4e" containerName="main"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798390    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="a158e7ca-9e93-4339-b291-59b594ad4a4e" containerName="main"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798464    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="a158e7ca-9e93-4339-b291-59b594ad4a4e" containerName="main"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798475    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy"
Jul 06 19:38:29.798540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.798481    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbbfc107-c8cc-453e-8488-7851e1e9356c" containerName="istio-proxy"
Jul 06 19:38:29.801342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.801326    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.805485 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.805464    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 19:38:29.815750 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.815729    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"]
Jul 06 19:38:29.920973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.920937    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-credential-socket\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921144 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.920980    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qs8tn\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-kube-api-access-qs8tn\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921144 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.921031    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-envoy\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921144 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.921060    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-socket\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921144 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.921081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-certs\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921371 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.921149    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istiod-ca-cert\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921371 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.921221    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-podinfo\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921371 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.921259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-token\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:29.921371 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:29.921293    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-data\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.021929 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.021894    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-podinfo\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.021929 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.021929    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-token\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022055    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-data\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022107    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-credential-socket\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022284 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022229    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qs8tn\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-kube-api-access-qs8tn\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022341 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022295    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-envoy\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022460    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-socket\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022481    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-data\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022497    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-certs\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022533    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-credential-socket\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022575    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istiod-ca-cert\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.022813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022805    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-socket\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.023087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.022866    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-certs\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.023330 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.023308    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istiod-ca-cert\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.024322 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.024301    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-podinfo\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.024549 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.024531    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-envoy\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.039913 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.039886    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qs8tn\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-kube-api-access-qs8tn\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.042047 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.042017    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-token\") pod \"e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") " pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.114484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.114406    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:30.255565 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.255539    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"]
Jul 06 19:38:30.256216 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:38:30.256193    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1e883f2_4da1_468e_9279_d0c0bc2c0b3a.slice/crio-d9c127c8b7015fa564efb8f9babdaa7edd025ad79919d724f38fce1f855d5dd1 WatchSource:0}: Error finding container d9c127c8b7015fa564efb8f9babdaa7edd025ad79919d724f38fce1f855d5dd1: Status 404 returned error can't find the container with id d9c127c8b7015fa564efb8f9babdaa7edd025ad79919d724f38fce1f855d5dd1
Jul 06 19:38:30.258168 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.258139    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:30.258234 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.258221    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:30.258275 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:30.258258    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:31.153794 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:31.153758    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" event={"ID":"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a","Type":"ContainerStarted","Data":"2a39690093f93973560344b545f54235726b304a2c4a44d2c9c79fa12f0a4244"}
Jul 06 19:38:31.154156 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:31.153811    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" event={"ID":"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a","Type":"ContainerStarted","Data":"d9c127c8b7015fa564efb8f9babdaa7edd025ad79919d724f38fce1f855d5dd1"}
Jul 06 19:38:31.181132 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:31.181082    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podStartSLOduration=2.181067475 podStartE2EDuration="2.181067475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:31.178373551 +0000 UTC m=+2370.789699102" watchObservedRunningTime="2026-07-06 19:38:31.181067475 +0000 UTC m=+2370.792393013"
Jul 06 19:38:32.115058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:32.115030    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:32.119399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:32.119374    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:32.158858 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:32.158821    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:32.159584 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:32.159567    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:38.623570 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.623539    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"]
Jul 06 19:38:38.626876 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.626860    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.667519 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.667491    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-7c8a24-openshift-default-dockercfg-8pf5r\""
Jul 06 19:38:38.672739 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.672712    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"]
Jul 06 19:38:38.791440 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791411    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istiod-ca-cert\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791607 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791455    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-envoy\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791607 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791497    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-certs\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791607 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791528    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-socket\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791607 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791543    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-token\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791847 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791612    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-podinfo\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791847 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791647    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-data\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791847 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791686    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-credential-socket\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.791847 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.791708    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqsxn\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-kube-api-access-mqsxn\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892214 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892150    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-envoy\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892214 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892187    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-certs\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892219    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-socket\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892246    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-token\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892286    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-podinfo\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892312    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-data\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892343    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-credential-socket\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892364    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqsxn\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-kube-api-access-mqsxn\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892400    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istiod-ca-cert\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892802 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892629    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-socket\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892802 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892697    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-certs\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892888 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892831    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-credential-socket\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.892930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.892911    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-data\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.893140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.893119    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istiod-ca-cert\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.894524 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.894502    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-envoy\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.894838 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.894819    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-podinfo\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.903340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.903316    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-token\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.903760 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.903743    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqsxn\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-kube-api-access-mqsxn\") pod \"e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") " pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:38.939507 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:38.939481    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:39.097307 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.097285    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"]
Jul 06 19:38:39.099186 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:38:39.099156    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec699b2c_e28f_411e_8d76_ced6c5c6e61a.slice/crio-fc316605555ae256dc7d9f1dd8f467c8f208934db520b5d61aff3214333f2eb4 WatchSource:0}: Error finding container fc316605555ae256dc7d9f1dd8f467c8f208934db520b5d61aff3214333f2eb4: Status 404 returned error can't find the container with id fc316605555ae256dc7d9f1dd8f467c8f208934db520b5d61aff3214333f2eb4
Jul 06 19:38:39.101180 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.101153    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:39.101254 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.101213    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:39.101254 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.101241    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:39.199817 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.199787    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" event={"ID":"ec699b2c-e28f-411e-8d76-ced6c5c6e61a","Type":"ContainerStarted","Data":"89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5"}
Jul 06 19:38:39.199951 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.199835    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" event={"ID":"ec699b2c-e28f-411e-8d76-ced6c5c6e61a","Type":"ContainerStarted","Data":"fc316605555ae256dc7d9f1dd8f467c8f208934db520b5d61aff3214333f2eb4"}
Jul 06 19:38:39.230524 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.230475    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" podStartSLOduration=1.230459862 podStartE2EDuration="1.230459862s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:39.229087448 +0000 UTC m=+2378.840412990" watchObservedRunningTime="2026-07-06 19:38:39.230459862 +0000 UTC m=+2378.841785469"
Jul 06 19:38:39.940208 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:39.940179    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:40.949964 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:40.949922    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:41.211149 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:41.211073    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:41.212072 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:41.212049    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:47.161587 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.161221    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 06 19:38:47.161587 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.161285    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 06 19:38:47.246134 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.246096    2579 generic.go:356] "Generic (PLEG): container finished" podID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerID="2a39690093f93973560344b545f54235726b304a2c4a44d2c9c79fa12f0a4244" exitCode=0
Jul 06 19:38:47.246287 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.246238    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" event={"ID":"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a","Type":"ContainerDied","Data":"2a39690093f93973560344b545f54235726b304a2c4a44d2c9c79fa12f0a4244"}
Jul 06 19:38:47.246702 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.246681    2579 scope.go:117] "RemoveContainer" containerID="2a39690093f93973560344b545f54235726b304a2c4a44d2c9c79fa12f0a4244"
Jul 06 19:38:47.248211 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.248169    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:47.248308 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.248288    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:47.248380 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:47.248339    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:48.278054 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:48.278013    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" event={"ID":"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a","Type":"ContainerStarted","Data":"d9351b89bd0b0be0eb2026b1c740a7db5733a8b04223285d9a59f5ee3e008198"}
Jul 06 19:38:49.115364 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:49.115317    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:49.115579 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:49.115551    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 06 19:38:49.115691 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:49.115629    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 06 19:38:50.115386 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:50.115346    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 06 19:38:50.115775 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:50.115411    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 06 19:38:51.115793 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.115754    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 06 19:38:51.116211 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.115811    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 06 19:38:51.783387 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.783352    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"]
Jul 06 19:38:51.785790 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.785770    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.788848 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.788824    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3d94e6\"/\"openshift-service-ca.crt\""
Jul 06 19:38:51.788961 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.788825    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3d94e6\"/\"default-dockercfg-s8k6l\""
Jul 06 19:38:51.790014 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.789995    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3d94e6\"/\"kube-root-ca.crt\""
Jul 06 19:38:51.790310 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.790288    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3d94e6\"/\"test-model-3d94e6-kserve-self-signed-certs\""
Jul 06 19:38:51.800686 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.800648    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"]
Jul 06 19:38:51.806708 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.806683    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c40015-dacf-4456-8898-c72e8e6f791b-tls-certs\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.806792 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.806721    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b96lp\" (UniqueName: \"kubernetes.io/projected/80c40015-dacf-4456-8898-c72e8e6f791b-kube-api-access-b96lp\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.806792 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.806783    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-tmp-dir\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.806882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.806860    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-home\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.806918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.806905    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-dshm\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.806953 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.806938    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-model-cache\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.907484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.907448    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c40015-dacf-4456-8898-c72e8e6f791b-tls-certs\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.907677 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.907491    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b96lp\" (UniqueName: \"kubernetes.io/projected/80c40015-dacf-4456-8898-c72e8e6f791b-kube-api-access-b96lp\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.907677 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.907625    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-tmp-dir\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.907804 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.907724    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-home\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.907804 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.907759    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-dshm\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.907804 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.907784    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-model-cache\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.908095 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.908030    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-tmp-dir\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.908095 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.908077    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-home\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.908228 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.908204    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-model-cache\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.909918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.909897    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-dshm\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.910009 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.909995    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c40015-dacf-4456-8898-c72e8e6f791b-tls-certs\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:51.916442 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:51.916422    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b96lp\" (UniqueName: \"kubernetes.io/projected/80c40015-dacf-4456-8898-c72e8e6f791b-kube-api-access-b96lp\") pod \"test-model-3d94e6-kserve-74f85b56c5-jhqzb\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") " pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:52.096209 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:52.096174    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:38:52.115706 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:52.115675    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body=
Jul 06 19:38:52.115830 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:52.115750    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused"
Jul 06 19:38:52.239972 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:52.239945    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"]
Jul 06 19:38:52.242400 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:38:52.242371    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80c40015_dacf_4456_8898_c72e8e6f791b.slice/crio-10167189cbc20fdce372d707e7f5bbe22f1413ae9aa4d876bc59fd9258e72de5 WatchSource:0}: Error finding container 10167189cbc20fdce372d707e7f5bbe22f1413ae9aa4d876bc59fd9258e72de5: Status 404 returned error can't find the container with id 10167189cbc20fdce372d707e7f5bbe22f1413ae9aa4d876bc59fd9258e72de5
Jul 06 19:38:52.302417 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:52.302387    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb" event={"ID":"80c40015-dacf-4456-8898-c72e8e6f791b","Type":"ContainerStarted","Data":"efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266"}
Jul 06 19:38:52.302546 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:52.302436    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb" event={"ID":"80c40015-dacf-4456-8898-c72e8e6f791b","Type":"ContainerStarted","Data":"10167189cbc20fdce372d707e7f5bbe22f1413ae9aa4d876bc59fd9258e72de5"}
Jul 06 19:38:52.324292 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:52.324244    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb" podStartSLOduration=1.324228113 podStartE2EDuration="1.324228113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:38:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:38:52.323320117 +0000 UTC m=+2391.934645657" watchObservedRunningTime="2026-07-06 19:38:52.324228113 +0000 UTC m=+2391.935553657"
Jul 06 19:38:53.119004 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.118930    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:53.119432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.119187    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:53.119806 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.119790    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:38:53.308498 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.308468    2579 generic.go:356] "Generic (PLEG): container finished" podID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerID="89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5" exitCode=0
Jul 06 19:38:53.308727 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.308542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" event={"ID":"ec699b2c-e28f-411e-8d76-ced6c5c6e61a","Type":"ContainerDied","Data":"89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5"}
Jul 06 19:38:53.309450 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.309430    2579 scope.go:117] "RemoveContainer" containerID="89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5"
Jul 06 19:38:53.310461 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.310419    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:53.310537 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.310515    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:53.310585 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:53.310569    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:38:54.315442 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:54.315401    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" event={"ID":"ec699b2c-e28f-411e-8d76-ced6c5c6e61a","Type":"ContainerStarted","Data":"c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63"}
Jul 06 19:38:54.940585 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:54.940545    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:54.941885 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:54.941852    2579 patch_prober.go:28] interesting pod/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 06 19:38:54.942030 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:54.941932    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 06 19:38:55.940276 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:55.940236    2579 patch_prober.go:28] interesting pod/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 06 19:38:55.940782 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:55.940299    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 06 19:38:56.212116 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:56.212020    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:56.940409 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:56.940376    2579 patch_prober.go:28] interesting pod/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 06 19:38:56.941017 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:56.940985    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 06 19:38:57.940780 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:57.940737    2579 patch_prober.go:28] interesting pod/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Jul 06 19:38:57.941304 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:57.940804    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Jul 06 19:38:58.943930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:58.943903    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:38:58.945061 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:38:58.945036    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:39:02.096351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:02.096315    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:39:02.096351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:02.096359    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:39:02.108711 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:02.108688    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:39:02.371874 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:02.371797    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:39:22.398351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.398308    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"]
Jul 06 19:39:22.402128 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.402102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.404886 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.404862    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7c8a24\"/\"test-model-7c8a24-kserve-self-signed-certs\""
Jul 06 19:39:22.406515 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.406490    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7c8a24\"/\"kube-root-ca.crt\""
Jul 06 19:39:22.406612 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.406515    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7c8a24\"/\"openshift-service-ca.crt\""
Jul 06 19:39:22.406612 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.406495    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7c8a24\"/\"default-dockercfg-lmjj4\""
Jul 06 19:39:22.409917 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.409896    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"]
Jul 06 19:39:22.443133 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.443109    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tmp-dir\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.443242 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.443146    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-dshm\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.443242 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.443166    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lnsp\" (UniqueName: \"kubernetes.io/projected/4dcd8fd9-7337-4832-baa6-b12da15d3abc-kube-api-access-6lnsp\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.443242 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.443195    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-model-cache\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.443242 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.443237    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tls-certs\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.443478 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.443360    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-home\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.544431 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544404    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-dshm\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.544576 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544434    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lnsp\" (UniqueName: \"kubernetes.io/projected/4dcd8fd9-7337-4832-baa6-b12da15d3abc-kube-api-access-6lnsp\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.544576 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544454    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-model-cache\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.544576 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544473    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tls-certs\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.544576 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544511    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-home\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.544576 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544554    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tmp-dir\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.544950 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544919    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-model-cache\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.545051 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544957    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tmp-dir\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.545051 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.544988    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-home\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.546724 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.546698    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-dshm\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.547003 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.546987    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tls-certs\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.553887 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.553865    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lnsp\" (UniqueName: \"kubernetes.io/projected/4dcd8fd9-7337-4832-baa6-b12da15d3abc-kube-api-access-6lnsp\") pod \"test-model-7c8a24-kserve-76777ff67f-6996z\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") " pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.712182 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.712106    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:22.848908 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:22.848882    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"]
Jul 06 19:39:22.850709 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:39:22.850681    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4dcd8fd9_7337_4832_baa6_b12da15d3abc.slice/crio-7fc362ffdad5dcb7cdade566c5e3f8d8fc86b0e22c82b954bff98ca99f00ad58 WatchSource:0}: Error finding container 7fc362ffdad5dcb7cdade566c5e3f8d8fc86b0e22c82b954bff98ca99f00ad58: Status 404 returned error can't find the container with id 7fc362ffdad5dcb7cdade566c5e3f8d8fc86b0e22c82b954bff98ca99f00ad58
Jul 06 19:39:23.470077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:23.470044    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z" event={"ID":"4dcd8fd9-7337-4832-baa6-b12da15d3abc","Type":"ContainerStarted","Data":"6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b"}
Jul 06 19:39:23.470446 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:23.470085    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z" event={"ID":"4dcd8fd9-7337-4832-baa6-b12da15d3abc","Type":"ContainerStarted","Data":"7fc362ffdad5dcb7cdade566c5e3f8d8fc86b0e22c82b954bff98ca99f00ad58"}
Jul 06 19:39:23.493223 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:23.493177    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z" podStartSLOduration=1.493163263 podStartE2EDuration="1.493163263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:39:23.492972081 +0000 UTC m=+2423.104297619" watchObservedRunningTime="2026-07-06 19:39:23.493163263 +0000 UTC m=+2423.104488800"
Jul 06 19:39:32.713136 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:32.713089    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:32.713775 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:32.713148    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:32.728745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:32.728720    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:33.536560 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:33.536529    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:39:47.110920 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:47.110886    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"]
Jul 06 19:39:47.991856 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:39:47.991825    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"]
Jul 06 19:40:02.130474 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.130413    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb" podUID="80c40015-dacf-4456-8898-c72e8e6f791b" containerName="main" containerID="cri-o://efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266" gracePeriod=45
Jul 06 19:40:02.378918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.378897    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:40:02.459563 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459505    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-model-cache\") pod \"80c40015-dacf-4456-8898-c72e8e6f791b\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") "
Jul 06 19:40:02.459563 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459547    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-home\") pod \"80c40015-dacf-4456-8898-c72e8e6f791b\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") "
Jul 06 19:40:02.459782 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459581    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c40015-dacf-4456-8898-c72e8e6f791b-tls-certs\") pod \"80c40015-dacf-4456-8898-c72e8e6f791b\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") "
Jul 06 19:40:02.459782 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459604    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b96lp\" (UniqueName: \"kubernetes.io/projected/80c40015-dacf-4456-8898-c72e8e6f791b-kube-api-access-b96lp\") pod \"80c40015-dacf-4456-8898-c72e8e6f791b\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") "
Jul 06 19:40:02.459782 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459635    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-tmp-dir\") pod \"80c40015-dacf-4456-8898-c72e8e6f791b\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") "
Jul 06 19:40:02.459782 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459687    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-dshm\") pod \"80c40015-dacf-4456-8898-c72e8e6f791b\" (UID: \"80c40015-dacf-4456-8898-c72e8e6f791b\") "
Jul 06 19:40:02.459988 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459819    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-model-cache" (OuterVolumeSpecName: "model-cache") pod "80c40015-dacf-4456-8898-c72e8e6f791b" (UID: "80c40015-dacf-4456-8898-c72e8e6f791b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:02.459988 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459832    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-home" (OuterVolumeSpecName: "home") pod "80c40015-dacf-4456-8898-c72e8e6f791b" (UID: "80c40015-dacf-4456-8898-c72e8e6f791b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:02.459988 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459923    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "80c40015-dacf-4456-8898-c72e8e6f791b" (UID: "80c40015-dacf-4456-8898-c72e8e6f791b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:02.459988 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459968    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:02.459988 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.459985    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:02.461880 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.461857    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80c40015-dacf-4456-8898-c72e8e6f791b-kube-api-access-b96lp" (OuterVolumeSpecName: "kube-api-access-b96lp") pod "80c40015-dacf-4456-8898-c72e8e6f791b" (UID: "80c40015-dacf-4456-8898-c72e8e6f791b"). InnerVolumeSpecName "kube-api-access-b96lp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:40:02.461964 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.461879    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-dshm" (OuterVolumeSpecName: "dshm") pod "80c40015-dacf-4456-8898-c72e8e6f791b" (UID: "80c40015-dacf-4456-8898-c72e8e6f791b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:02.461964 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.461931    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80c40015-dacf-4456-8898-c72e8e6f791b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "80c40015-dacf-4456-8898-c72e8e6f791b" (UID: "80c40015-dacf-4456-8898-c72e8e6f791b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:40:02.560958 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.560932    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b96lp\" (UniqueName: \"kubernetes.io/projected/80c40015-dacf-4456-8898-c72e8e6f791b-kube-api-access-b96lp\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:02.561071 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.560970    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:02.561071 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.560981    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/80c40015-dacf-4456-8898-c72e8e6f791b-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:02.561071 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.560989    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/80c40015-dacf-4456-8898-c72e8e6f791b-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:02.670216 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.670180    2579 generic.go:356] "Generic (PLEG): container finished" podID="80c40015-dacf-4456-8898-c72e8e6f791b" containerID="efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266" exitCode=0
Jul 06 19:40:02.670342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.670220    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb" event={"ID":"80c40015-dacf-4456-8898-c72e8e6f791b","Type":"ContainerDied","Data":"efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266"}
Jul 06 19:40:02.670342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.670222    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"
Jul 06 19:40:02.670342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.670248    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb" event={"ID":"80c40015-dacf-4456-8898-c72e8e6f791b","Type":"ContainerDied","Data":"10167189cbc20fdce372d707e7f5bbe22f1413ae9aa4d876bc59fd9258e72de5"}
Jul 06 19:40:02.670342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.670269    2579 scope.go:117] "RemoveContainer" containerID="efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266"
Jul 06 19:40:02.681748 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.681686    2579 scope.go:117] "RemoveContainer" containerID="efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266"
Jul 06 19:40:02.682091 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:40:02.682070    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266\": container with ID starting with efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266 not found: ID does not exist" containerID="efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266"
Jul 06 19:40:02.682160 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.682099    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266"} err="failed to get container status \"efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266\": rpc error: code = NotFound desc = could not find container \"efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266\": container with ID starting with efa8ac3eefdbc62ae5dac53c7e5903b69bbf5c37a9a1763831dc1d4266ed2266 not found: ID does not exist"
Jul 06 19:40:02.696484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.696464    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"]
Jul 06 19:40:02.703313 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.703295    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb"]
Jul 06 19:40:02.927612 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:02.927583    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80c40015-dacf-4456-8898-c72e8e6f791b" path="/var/lib/kubelet/pods/80c40015-dacf-4456-8898-c72e8e6f791b/volumes"
Jul 06 19:40:03.012743 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.012717    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z" podUID="4dcd8fd9-7337-4832-baa6-b12da15d3abc" containerName="main" containerID="cri-o://6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b" gracePeriod=45
Jul 06 19:40:03.259077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.259056    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:40:03.367026 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.366999    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tls-certs\") pod \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") "
Jul 06 19:40:03.367184 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367084    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-home\") pod \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") "
Jul 06 19:40:03.367184 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367110    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lnsp\" (UniqueName: \"kubernetes.io/projected/4dcd8fd9-7337-4832-baa6-b12da15d3abc-kube-api-access-6lnsp\") pod \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") "
Jul 06 19:40:03.367184 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367129    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tmp-dir\") pod \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") "
Jul 06 19:40:03.367184 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367163    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-model-cache\") pod \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") "
Jul 06 19:40:03.367411 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367206    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-dshm\") pod \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\" (UID: \"4dcd8fd9-7337-4832-baa6-b12da15d3abc\") "
Jul 06 19:40:03.367411 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367378    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4dcd8fd9-7337-4832-baa6-b12da15d3abc" (UID: "4dcd8fd9-7337-4832-baa6-b12da15d3abc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:03.367411 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367384    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-home" (OuterVolumeSpecName: "home") pod "4dcd8fd9-7337-4832-baa6-b12da15d3abc" (UID: "4dcd8fd9-7337-4832-baa6-b12da15d3abc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:03.367588 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367405    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-model-cache" (OuterVolumeSpecName: "model-cache") pod "4dcd8fd9-7337-4832-baa6-b12da15d3abc" (UID: "4dcd8fd9-7337-4832-baa6-b12da15d3abc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:03.367588 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367508    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:03.367588 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367520    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:03.367588 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.367535    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:03.369351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.369323    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-dshm" (OuterVolumeSpecName: "dshm") pod "4dcd8fd9-7337-4832-baa6-b12da15d3abc" (UID: "4dcd8fd9-7337-4832-baa6-b12da15d3abc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:40:03.369351 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.369334    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4dcd8fd9-7337-4832-baa6-b12da15d3abc" (UID: "4dcd8fd9-7337-4832-baa6-b12da15d3abc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:40:03.369486 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.369374    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4dcd8fd9-7337-4832-baa6-b12da15d3abc-kube-api-access-6lnsp" (OuterVolumeSpecName: "kube-api-access-6lnsp") pod "4dcd8fd9-7337-4832-baa6-b12da15d3abc" (UID: "4dcd8fd9-7337-4832-baa6-b12da15d3abc"). InnerVolumeSpecName "kube-api-access-6lnsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:40:03.468494 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.468435    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4dcd8fd9-7337-4832-baa6-b12da15d3abc-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:03.468494 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.468457    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lnsp\" (UniqueName: \"kubernetes.io/projected/4dcd8fd9-7337-4832-baa6-b12da15d3abc-kube-api-access-6lnsp\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:03.468494 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.468473    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4dcd8fd9-7337-4832-baa6-b12da15d3abc-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:40:03.676263 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.676230    2579 generic.go:356] "Generic (PLEG): container finished" podID="4dcd8fd9-7337-4832-baa6-b12da15d3abc" containerID="6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b" exitCode=0
Jul 06 19:40:03.676413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.676273    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"
Jul 06 19:40:03.676413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.676312    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z" event={"ID":"4dcd8fd9-7337-4832-baa6-b12da15d3abc","Type":"ContainerDied","Data":"6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b"}
Jul 06 19:40:03.676413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.676355    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z" event={"ID":"4dcd8fd9-7337-4832-baa6-b12da15d3abc","Type":"ContainerDied","Data":"7fc362ffdad5dcb7cdade566c5e3f8d8fc86b0e22c82b954bff98ca99f00ad58"}
Jul 06 19:40:03.676413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.676373    2579 scope.go:117] "RemoveContainer" containerID="6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b"
Jul 06 19:40:03.685485 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.685469    2579 scope.go:117] "RemoveContainer" containerID="6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b"
Jul 06 19:40:03.685740 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:40:03.685720    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b\": container with ID starting with 6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b not found: ID does not exist" containerID="6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b"
Jul 06 19:40:03.685798 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.685750    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b"} err="failed to get container status \"6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b\": rpc error: code = NotFound desc = could not find container \"6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b\": container with ID starting with 6c2dd30db08b3e388503e8bb0b681f772ed4406fd1fa13357d2e873fe9145f0b not found: ID does not exist"
Jul 06 19:40:03.701388 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.701366    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"]
Jul 06 19:40:03.711319 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:03.711300    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7c8a24/test-model-7c8a24-kserve-76777ff67f-6996z"]
Jul 06 19:40:04.928745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:04.928712    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4dcd8fd9-7337-4832-baa6-b12da15d3abc" path="/var/lib/kubelet/pods/4dcd8fd9-7337-4832-baa6-b12da15d3abc/volumes"
Jul 06 19:40:07.361761 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:40:07.361719    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-inf-a-3d94e6/test-model-3d94e6-kserve-74f85b56c5-jhqzb" podUID="80c40015-dacf-4456-8898-c72e8e6f791b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.53:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:41:18.957098 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:18.957060    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"]
Jul 06 19:41:18.957585 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:18.957265    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy" containerID="cri-o://d9351b89bd0b0be0eb2026b1c740a7db5733a8b04223285d9a59f5ee3e008198" gracePeriod=30
Jul 06 19:41:23.121579 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:23.121533    2579 patch_prober.go:28] interesting pod/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 19:41:23.122040 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:23.121614    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 19:41:24.098912 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.098876    2579 generic.go:356] "Generic (PLEG): container finished" podID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerID="d9351b89bd0b0be0eb2026b1c740a7db5733a8b04223285d9a59f5ee3e008198" exitCode=0
Jul 06 19:41:24.099176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.098972    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" event={"ID":"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a","Type":"ContainerDied","Data":"d9351b89bd0b0be0eb2026b1c740a7db5733a8b04223285d9a59f5ee3e008198"}
Jul 06 19:41:24.099387 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.099374    2579 scope.go:117] "RemoveContainer" containerID="2a39690093f93973560344b545f54235726b304a2c4a44d2c9c79fa12f0a4244"
Jul 06 19:41:24.211919 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.211897    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:41:24.306940 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.306913    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-credential-socket\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.306961    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-socket\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.306988    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istiod-ca-cert\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307050    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-envoy\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307074    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qs8tn\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-kube-api-access-qs8tn\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307099    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-certs\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307144    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-podinfo\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307174    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-data\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307205    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:41:24.307243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307220    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:41:24.307243 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307223    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-token\") pod \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\" (UID: \"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a\") "
Jul 06 19:41:24.307602 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307458    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:41:24.307602 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307542    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:41:24.307745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307633    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.307745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307647    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.307745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307677    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.307745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.307690    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.308109 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.308084    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:41:24.309151 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.309123    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:41:24.309712 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.309678    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:41:24.309712 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.309705    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:41:24.309829 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.309684    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-kube-api-access-qs8tn" (OuterVolumeSpecName: "kube-api-access-qs8tn") pod "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" (UID: "b1e883f2-4da1-468e-9279-d0c0bc2c0b3a"). InnerVolumeSpecName "kube-api-access-qs8tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:41:24.409017 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.408988    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.409017 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.409013    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.409017 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.409022    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.409222 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.409030    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:24.409222 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:24.409039    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qs8tn\" (UniqueName: \"kubernetes.io/projected/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a-kube-api-access-qs8tn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:41:25.105737 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:25.105704    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm" event={"ID":"b1e883f2-4da1-468e-9279-d0c0bc2c0b3a","Type":"ContainerDied","Data":"d9c127c8b7015fa564efb8f9babdaa7edd025ad79919d724f38fce1f855d5dd1"}
Jul 06 19:41:25.105944 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:25.105763    2579 scope.go:117] "RemoveContainer" containerID="d9351b89bd0b0be0eb2026b1c740a7db5733a8b04223285d9a59f5ee3e008198"
Jul 06 19:41:25.105944 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:25.105784    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"
Jul 06 19:41:29.972796 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:29.972762    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"]
Jul 06 19:41:29.975339 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:29.975314    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-3d94e6-openshift-default-676444fcf-hvdfm"]
Jul 06 19:41:30.927822 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:30.927781    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" path="/var/lib/kubelet/pods/b1e883f2-4da1-468e-9279-d0c0bc2c0b3a/volumes"
Jul 06 19:41:41.433046 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:41.433003    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7558657f96-wwrjg" podUID="457f7f2d-3135-4e8c-9fcd-d8d0641a8314" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.42:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:41:56.959423 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:56.959388    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"]
Jul 06 19:41:56.960042 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:56.959651    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy" containerID="cri-o://c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63" gracePeriod=30
Jul 06 19:41:58.946742 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:58.946706    2579 patch_prober.go:28] interesting pod/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 06 19:41:58.947100 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:41:58.946774    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 06 19:42:02.210085 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.210061    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:42:02.298026 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.297995    2579 generic.go:356] "Generic (PLEG): container finished" podID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerID="c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63" exitCode=0
Jul 06 19:42:02.298166 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.298040    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" event={"ID":"ec699b2c-e28f-411e-8d76-ced6c5c6e61a","Type":"ContainerDied","Data":"c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63"}
Jul 06 19:42:02.298166 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.298048    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"
Jul 06 19:42:02.298166 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.298112    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l" event={"ID":"ec699b2c-e28f-411e-8d76-ced6c5c6e61a","Type":"ContainerDied","Data":"fc316605555ae256dc7d9f1dd8f467c8f208934db520b5d61aff3214333f2eb4"}
Jul 06 19:42:02.298166 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.298135    2579 scope.go:117] "RemoveContainer" containerID="c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63"
Jul 06 19:42:02.306722 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.306702    2579 scope.go:117] "RemoveContainer" containerID="89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5"
Jul 06 19:42:02.310051 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310031    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-envoy\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310164 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310113    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-token\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310164 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310138    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-socket\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310164 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310159    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-credential-socket\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310334 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310197    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istiod-ca-cert\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310334 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310229    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mqsxn\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-kube-api-access-mqsxn\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310334 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310263    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-data\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310334 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310299    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-podinfo\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310528 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310335    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-certs\") pod \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\" (UID: \"ec699b2c-e28f-411e-8d76-ced6c5c6e61a\") "
Jul 06 19:42:02.310528 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310487    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:42:02.310528 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310500    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:42:02.311025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310714    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:42:02.311025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310853    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:42:02.311025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310938    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.311025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310964    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.311025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310979    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.311025 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.310996    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.311393 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.311106    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-data" (OuterVolumeSpecName: "istio-data") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:42:02.312445 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.312421    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:42:02.312614 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.312591    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:42:02.312921 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.312894    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:42:02.312921 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.312905    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-kube-api-access-mqsxn" (OuterVolumeSpecName: "kube-api-access-mqsxn") pod "ec699b2c-e28f-411e-8d76-ced6c5c6e61a" (UID: "ec699b2c-e28f-411e-8d76-ced6c5c6e61a"). InnerVolumeSpecName "kube-api-access-mqsxn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:42:02.316640 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.316622    2579 scope.go:117] "RemoveContainer" containerID="c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63"
Jul 06 19:42:02.316968 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:42:02.316948    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63\": container with ID starting with c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63 not found: ID does not exist" containerID="c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63"
Jul 06 19:42:02.317064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.316979    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63"} err="failed to get container status \"c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63\": rpc error: code = NotFound desc = could not find container \"c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63\": container with ID starting with c7197dcb166f983a74004f8bc3c1d606e5692bc2b1a508fa488463d79cf94b63 not found: ID does not exist"
Jul 06 19:42:02.317064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.317003    2579 scope.go:117] "RemoveContainer" containerID="89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5"
Jul 06 19:42:02.317281 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:42:02.317264    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5\": container with ID starting with 89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5 not found: ID does not exist" containerID="89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5"
Jul 06 19:42:02.317321 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.317287    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5"} err="failed to get container status \"89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5\": rpc error: code = NotFound desc = could not find container \"89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5\": container with ID starting with 89a595b8ccc75d8ef62da81956dafcde8fbb8a92e7314709058c1489cc86aef5 not found: ID does not exist"
Jul 06 19:42:02.412154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.412127    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.412154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.412150    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mqsxn\" (UniqueName: \"kubernetes.io/projected/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-kube-api-access-mqsxn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.412310 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.412161    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.412310 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.412169    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.412310 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.412177    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ec699b2c-e28f-411e-8d76-ced6c5c6e61a-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:42:02.645022 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.644990    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"]
Jul 06 19:42:02.651945 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.651924    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-7c8a24-openshift-default-5788bbcfcc-fxt6l"]
Jul 06 19:42:02.931946 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:02.928636    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" path="/var/lib/kubelet/pods/ec699b2c-e28f-411e-8d76-ced6c5c6e61a/volumes"
Jul 06 19:42:04.428399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428361    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"]
Jul 06 19:42:04.428834 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428819    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy"
Jul 06 19:42:04.428881 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428837    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy"
Jul 06 19:42:04.428881 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428853    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80c40015-dacf-4456-8898-c72e8e6f791b" containerName="main"
Jul 06 19:42:04.428881 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428861    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="80c40015-dacf-4456-8898-c72e8e6f791b" containerName="main"
Jul 06 19:42:04.428881 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428871    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4dcd8fd9-7337-4832-baa6-b12da15d3abc" containerName="main"
Jul 06 19:42:04.428881 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428879    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="4dcd8fd9-7337-4832-baa6-b12da15d3abc" containerName="main"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428893    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428901    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428916    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428924    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.428994    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.429004    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="4dcd8fd9-7337-4832-baa6-b12da15d3abc" containerName="main"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.429013    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy"
Jul 06 19:42:04.429034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.429020    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="80c40015-dacf-4456-8898-c72e8e6f791b" containerName="main"
Jul 06 19:42:04.433763 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.433745    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.472552 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.472523    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"]
Jul 06 19:42:04.525863 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.525833    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-data\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.525994 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.525873    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istiod-ca-cert\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.525994 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.525906    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-socket\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.525994 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.525940    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-podinfo\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.525994 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.525969    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-token\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.526152 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.526010    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlch6\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-kube-api-access-jlch6\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.526152 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.526044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-credential-socket\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.526152 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.526075    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-envoy\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.526152 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.526099    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-certs\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627232 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627204    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlch6\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-kube-api-access-jlch6\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627241    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-credential-socket\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627263    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-envoy\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627282    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-certs\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-data\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627339    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istiod-ca-cert\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627385 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627363    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-socket\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627740 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627477    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-podinfo\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627740 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627528    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-token\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627740 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627645    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-credential-socket\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627897 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627878    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-socket\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.627968 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.627878    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-data\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.628077 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.628057    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-certs\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.629559 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.629540    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-envoy\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.629967 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.629944    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-podinfo\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.891199 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.891168    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 19:42:04.898214 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.898179    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istiod-ca-cert\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.898424 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.898401    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlch6\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-kube-api-access-jlch6\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:04.898464 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:04.898427    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-token\") pod \"e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") " pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:05.043177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.043149    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:05.210713 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.210691    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"]
Jul 06 19:42:05.216632 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.216604    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:05.216762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.216689    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:05.216762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.216735    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:05.317420 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.317386    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" event={"ID":"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f","Type":"ContainerStarted","Data":"e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b"}
Jul 06 19:42:05.317601 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.317438    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" event={"ID":"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f","Type":"ContainerStarted","Data":"d27d59058ffdcdff7e37c65e0d07863bc3f07f150ad17cb0cb3cfa433c2a937f"}
Jul 06 19:42:05.351506 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:05.351450    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" podStartSLOduration=1.351434226 podStartE2EDuration="1.351434226s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:42:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:42:05.34927138 +0000 UTC m=+2584.960596943" watchObservedRunningTime="2026-07-06 19:42:05.351434226 +0000 UTC m=+2584.962759823"
Jul 06 19:42:06.043771 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:06.043736    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:06.048623 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:06.048598    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:06.323228 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:06.323148    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:06.323946 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:06.323927    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:11.433930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:11.433883    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7558657f96-wwrjg" podUID="457f7f2d-3135-4e8c-9fcd-d8d0641a8314" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.42:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:42:31.433671 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:31.433574    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-controller-7558657f96-wwrjg" podUID="457f7f2d-3135-4e8c-9fcd-d8d0641a8314" containerName="manager" probeResult="failure" output="Get \"http://10.133.0.42:8081/readyz\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:42:38.218895 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.218864    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"]
Jul 06 19:42:38.219257 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.219208    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy"
Jul 06 19:42:38.219257 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.219219    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy"
Jul 06 19:42:38.219329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.219278    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1e883f2-4da1-468e-9279-d0c0bc2c0b3a" containerName="istio-proxy"
Jul 06 19:42:38.219329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.219287    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec699b2c-e28f-411e-8d76-ced6c5c6e61a" containerName="istio-proxy"
Jul 06 19:42:38.222823 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.222787    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294227 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294194    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-certs\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294227 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294226    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-envoy\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294455 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294245    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-data\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294455 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-socket\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294455 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294368    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-credential-socket\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294455 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294409    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-token\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294684 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294488    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-podinfo\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294684 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294522    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpbbn\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-kube-api-access-qpbbn\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.294684 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.294542    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istiod-ca-cert\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.326848 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.326821    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"]
Jul 06 19:42:38.395080 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395057    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-certs\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395084    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-envoy\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395103    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-data\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395127    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-socket\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-credential-socket\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395317    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-token\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395376    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-podinfo\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395399    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpbbn\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-kube-api-access-qpbbn\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395420    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istiod-ca-cert\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395686 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395644    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-certs\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.395889 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.395866    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-socket\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.396094 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.396067    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-data\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.396218 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.396090    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-credential-socket\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.396218 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.396172    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istiod-ca-cert\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.397500 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.397472    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-envoy\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.397959 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.397940    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-podinfo\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.478817 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.478712    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-token\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.487373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.487351    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpbbn\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-kube-api-access-qpbbn\") pod \"e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") " pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.533248 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.533226    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:38.712110 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.712083    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"]
Jul 06 19:42:38.712243 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:42:38.712158    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d26c5f2_79bc_493a_bf63_13035d7a22c9.slice/crio-b2893022849fe97477c2ab2890b3f17e2cbe712941417ab942354721e04f277f WatchSource:0}: Error finding container b2893022849fe97477c2ab2890b3f17e2cbe712941417ab942354721e04f277f: Status 404 returned error can't find the container with id b2893022849fe97477c2ab2890b3f17e2cbe712941417ab942354721e04f277f
Jul 06 19:42:38.714282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.714256    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:38.714347 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.714328    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:38.714386 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:38.714356    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:39.487216 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:39.487176    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" event={"ID":"6d26c5f2-79bc-493a-bf63-13035d7a22c9","Type":"ContainerStarted","Data":"4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889"}
Jul 06 19:42:39.487586 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:39.487231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" event={"ID":"6d26c5f2-79bc-493a-bf63-13035d7a22c9","Type":"ContainerStarted","Data":"b2893022849fe97477c2ab2890b3f17e2cbe712941417ab942354721e04f277f"}
Jul 06 19:42:39.534030 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:39.534000    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:39.538794 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:39.538772    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:39.561476 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:39.561434    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" podStartSLOduration=1.561420519 podStartE2EDuration="1.561420519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:42:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:42:39.559473863 +0000 UTC m=+2619.170799403" watchObservedRunningTime="2026-07-06 19:42:39.561420519 +0000 UTC m=+2619.172746057"
Jul 06 19:42:40.492291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:40.492258    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:40.493201 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:40.493181    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:42:51.324667 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.324617    2579 patch_prober.go:28] interesting pod/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 19:42:51.325161 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.324699    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 19:42:51.557049 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.556090    2579 generic.go:356] "Generic (PLEG): container finished" podID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerID="e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b" exitCode=0
Jul 06 19:42:51.557049 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.556174    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" event={"ID":"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f","Type":"ContainerDied","Data":"e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b"}
Jul 06 19:42:51.557049 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.556675    2579 scope.go:117] "RemoveContainer" containerID="e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b"
Jul 06 19:42:51.559234 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.559186    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:51.559381 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.559350    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:51.559475 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:51.559453    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:52.566413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:52.566366    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" event={"ID":"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f","Type":"ContainerStarted","Data":"6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f"}
Jul 06 19:42:53.044331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:53.044297    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:53.044607 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:53.044317    2579 patch_prober.go:28] interesting pod/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 19:42:53.044806 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:53.044782    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 19:42:54.043973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:54.043925    2579 patch_prober.go:28] interesting pod/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 19:42:54.044538 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:54.044001    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 19:42:55.043834 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:55.043796    2579 patch_prober.go:28] interesting pod/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 19:42:55.044043 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:55.043858    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 19:42:56.044298 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:56.044262    2579 patch_prober.go:28] interesting pod/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Jul 06 19:42:56.044890 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:56.044851    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Jul 06 19:42:56.533928 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:56.533825    2579 patch_prober.go:28] interesting pod/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 06 19:42:56.533928 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:56.533900    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 06 19:42:57.047342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:57.047312    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:57.047803 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:57.047600    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:57.048267 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:57.048249    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:42:58.610927 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:58.610854    2579 generic.go:356] "Generic (PLEG): container finished" podID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerID="4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889" exitCode=0
Jul 06 19:42:58.611282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:58.610932    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" event={"ID":"6d26c5f2-79bc-493a-bf63-13035d7a22c9","Type":"ContainerDied","Data":"4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889"}
Jul 06 19:42:58.611477 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:58.611458    2579 scope.go:117] "RemoveContainer" containerID="4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889"
Jul 06 19:42:58.612388 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:58.612363    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:58.612438 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:58.612428    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:58.612484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:58.612473    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:42:59.617585 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:42:59.617547    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" event={"ID":"6d26c5f2-79bc-493a-bf63-13035d7a22c9","Type":"ContainerStarted","Data":"499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e"}
Jul 06 19:43:00.533370 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:00.533340    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:43:00.533840 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:00.533812    2579 patch_prober.go:28] interesting pod/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 06 19:43:00.534154 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:00.534107    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 06 19:43:01.534705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:01.534170    2579 patch_prober.go:28] interesting pod/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 06 19:43:01.534705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:01.534232    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 06 19:43:02.534193 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:02.534147    2579 patch_prober.go:28] interesting pod/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 06 19:43:02.534505 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:02.534214    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 06 19:43:03.534405 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:03.534363    2579 patch_prober.go:28] interesting pod/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body=
Jul 06 19:43:03.534929 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:03.534440    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused"
Jul 06 19:43:04.537207 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:04.537180    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:43:04.537608 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:04.537584    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:43:04.538418 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:04.538397    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:43:12.109616 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:12.109576    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"]
Jul 06 19:43:12.110030 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:12.109830    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy" containerID="cri-o://6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f" gracePeriod=30
Jul 06 19:43:17.369153 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.369102    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:43:17.435878 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.435854    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-data\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.435878 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.435881    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-envoy\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.435920    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-token\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.435950    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-socket\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.435991    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlch6\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-kube-api-access-jlch6\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436034    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istiod-ca-cert\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436061    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-podinfo\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436076    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-credential-socket\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436104    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-certs\") pod \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\" (UID: \"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f\") "
Jul 06 19:43:17.436399 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436252    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:17.436557 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436464    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.436557 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436468    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:17.436737 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436712    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:17.436889 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436775    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:43:17.436889 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.436817    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-data" (OuterVolumeSpecName: "istio-data") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:17.438156 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.438126    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-kube-api-access-jlch6" (OuterVolumeSpecName: "kube-api-access-jlch6") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "kube-api-access-jlch6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:43:17.438156 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.438131    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:17.438317 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.438179    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-token" (OuterVolumeSpecName: "istio-token") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:43:17.438317 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.438185    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" (UID: "7fcfc4d8-9117-46cd-ab13-f6c25db8da3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:43:17.537095 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537069    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.537095 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537090    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.537226 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537099    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.537226 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537108    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.537226 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537116    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.537226 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537124    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.537226 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537131    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.537226 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.537138    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlch6\" (UniqueName: \"kubernetes.io/projected/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f-kube-api-access-jlch6\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:17.721069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.720998    2579 generic.go:356] "Generic (PLEG): container finished" podID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerID="6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f" exitCode=0
Jul 06 19:43:17.721069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.721054    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"
Jul 06 19:43:17.721224 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.721076    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" event={"ID":"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f","Type":"ContainerDied","Data":"6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f"}
Jul 06 19:43:17.721224 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.721159    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh" event={"ID":"7fcfc4d8-9117-46cd-ab13-f6c25db8da3f","Type":"ContainerDied","Data":"d27d59058ffdcdff7e37c65e0d07863bc3f07f150ad17cb0cb3cfa433c2a937f"}
Jul 06 19:43:17.721224 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.721184    2579 scope.go:117] "RemoveContainer" containerID="6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f"
Jul 06 19:43:17.730267 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.730252    2579 scope.go:117] "RemoveContainer" containerID="e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b"
Jul 06 19:43:17.737953 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.737936    2579 scope.go:117] "RemoveContainer" containerID="6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f"
Jul 06 19:43:17.738183 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:43:17.738166    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f\": container with ID starting with 6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f not found: ID does not exist" containerID="6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f"
Jul 06 19:43:17.738237 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.738191    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f"} err="failed to get container status \"6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f\": rpc error: code = NotFound desc = could not find container \"6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f\": container with ID starting with 6433ac5e7110c4117d8114aea92225710efaaf5c9a2aa744ee5707676f2b6a8f not found: ID does not exist"
Jul 06 19:43:17.738237 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.738208    2579 scope.go:117] "RemoveContainer" containerID="e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b"
Jul 06 19:43:17.738451 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:43:17.738432    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b\": container with ID starting with e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b not found: ID does not exist" containerID="e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b"
Jul 06 19:43:17.738488 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.738459    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b"} err="failed to get container status \"e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b\": rpc error: code = NotFound desc = could not find container \"e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b\": container with ID starting with e84f22e55af606dd1967967d3bc95f488ff94aee6627b75dbc768c113bdcfe6b not found: ID does not exist"
Jul 06 19:43:17.756348 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.756324    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"]
Jul 06 19:43:17.780394 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:17.780370    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-27c473-openshift-default-688bbf8b8c-cmvnh"]
Jul 06 19:43:18.926784 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:18.926753    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" path="/var/lib/kubelet/pods/7fcfc4d8-9117-46cd-ab13-f6c25db8da3f/volumes"
Jul 06 19:43:20.506291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:20.506254    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"]
Jul 06 19:43:20.506729 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:20.506465    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy" containerID="cri-o://499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e" gracePeriod=30
Jul 06 19:43:21.498041 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.498008    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"]
Jul 06 19:43:21.498336 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.498324    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy"
Jul 06 19:43:21.498336 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.498337    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy"
Jul 06 19:43:21.498433 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.498350    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy"
Jul 06 19:43:21.498433 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.498356    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy"
Jul 06 19:43:21.498433 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.498408    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy"
Jul 06 19:43:21.502849 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.502830    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.518870 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.518849    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"]
Jul 06 19:43:21.568817 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.568790    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wgzj\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-kube-api-access-7wgzj\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569047 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569024    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-data\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569160 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569081    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-credential-socket\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569160 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569123    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-certs\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569276 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569161    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-podinfo\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569276 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569220    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-envoy\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569276 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569260    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-token\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569437 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4c86e6-3a64-4227-89e4-da6961b83b50-istiod-ca-cert\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.569437 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.569325    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-socket\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670032 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670006    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-podinfo\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670165 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670042    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-envoy\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670165 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670075    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-token\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670165 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670098    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4c86e6-3a64-4227-89e4-da6961b83b50-istiod-ca-cert\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670165 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670130    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-socket\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670359 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670265    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wgzj\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-kube-api-access-7wgzj\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670359 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670327    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-data\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670485 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670374    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-credential-socket\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670485 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670410    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-certs\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670597 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670525    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-socket\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670729 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670708    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-data\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670807 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670738    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-certs\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.670807 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.670755    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-credential-socket\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.671193 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.671171    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4c86e6-3a64-4227-89e4-da6961b83b50-istiod-ca-cert\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.672614 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.672594    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-envoy\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.672786 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.672769    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-podinfo\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.685826 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.685800    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-token\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.685906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.685825    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wgzj\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-kube-api-access-7wgzj\") pod \"e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") " pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.812622 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.812597    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:21.961151 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.961122    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"]
Jul 06 19:43:21.962509 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:43:21.962484    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e4c86e6_3a64_4227_89e4_da6961b83b50.slice/crio-feab26b3669ae4a469bc8bbdca239749b3f9ffe309a331169baca0b46c061e37 WatchSource:0}: Error finding container feab26b3669ae4a469bc8bbdca239749b3f9ffe309a331169baca0b46c061e37: Status 404 returned error can't find the container with id feab26b3669ae4a469bc8bbdca239749b3f9ffe309a331169baca0b46c061e37
Jul 06 19:43:21.964631 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.964596    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:21.964746 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.964692    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:21.964746 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:21.964732    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:22.749110 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:22.749077    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" event={"ID":"7e4c86e6-3a64-4227-89e4-da6961b83b50","Type":"ContainerStarted","Data":"b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e"}
Jul 06 19:43:22.749463 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:22.749152    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" event={"ID":"7e4c86e6-3a64-4227-89e4-da6961b83b50","Type":"ContainerStarted","Data":"feab26b3669ae4a469bc8bbdca239749b3f9ffe309a331169baca0b46c061e37"}
Jul 06 19:43:22.773594 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:22.773548    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" podStartSLOduration=1.7735330299999998 podStartE2EDuration="1.77353303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:43:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:43:22.771868864 +0000 UTC m=+2662.383194402" watchObservedRunningTime="2026-07-06 19:43:22.77353303 +0000 UTC m=+2662.384858568"
Jul 06 19:43:22.813610 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:22.813588    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:23.818014 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:23.817977    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:24.760413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:24.760378    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:24.761273 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:24.761251    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:25.750535 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.750509    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:43:25.767848 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.767776    2579 generic.go:356] "Generic (PLEG): container finished" podID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerID="499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e" exitCode=0
Jul 06 19:43:25.767848 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.767820    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"
Jul 06 19:43:25.768020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.767859    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" event={"ID":"6d26c5f2-79bc-493a-bf63-13035d7a22c9","Type":"ContainerDied","Data":"499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e"}
Jul 06 19:43:25.768020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.767908    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh" event={"ID":"6d26c5f2-79bc-493a-bf63-13035d7a22c9","Type":"ContainerDied","Data":"b2893022849fe97477c2ab2890b3f17e2cbe712941417ab942354721e04f277f"}
Jul 06 19:43:25.768020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.767936    2579 scope.go:117] "RemoveContainer" containerID="499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e"
Jul 06 19:43:25.777434 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.777416    2579 scope.go:117] "RemoveContainer" containerID="4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889"
Jul 06 19:43:25.787218 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.787201    2579 scope.go:117] "RemoveContainer" containerID="499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e"
Jul 06 19:43:25.787480 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:43:25.787461    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e\": container with ID starting with 499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e not found: ID does not exist" containerID="499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e"
Jul 06 19:43:25.787527 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.787489    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e"} err="failed to get container status \"499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e\": rpc error: code = NotFound desc = could not find container \"499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e\": container with ID starting with 499f899a666b2510afb43a49e8100dbe01df4dfb7bb0958d5bfa6d5f0832973e not found: ID does not exist"
Jul 06 19:43:25.787527 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.787507    2579 scope.go:117] "RemoveContainer" containerID="4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889"
Jul 06 19:43:25.787782 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:43:25.787764    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889\": container with ID starting with 4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889 not found: ID does not exist" containerID="4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889"
Jul 06 19:43:25.787834 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.787788    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889"} err="failed to get container status \"4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889\": rpc error: code = NotFound desc = could not find container \"4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889\": container with ID starting with 4fa3a40aba4984bccecae7729c58ecd5cc7e07158497a3a9200d59ba8608a889 not found: ID does not exist"
Jul 06 19:43:25.908698 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908648    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-socket\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908698 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908704    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-certs\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908722    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-credential-socket\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908739    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-token\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908754    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-podinfo\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908773    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpbbn\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-kube-api-access-qpbbn\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908791    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istiod-ca-cert\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908851    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-envoy\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908877    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:25.908930 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.908902    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-data\") pod \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\" (UID: \"6d26c5f2-79bc-493a-bf63-13035d7a22c9\") "
Jul 06 19:43:25.909408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.909030    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:25.909408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.909046    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:25.909408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.909205    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:25.909408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.909226    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:25.909408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.909241    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:25.909408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.909297    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:43:25.909847 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.909823    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:25.911286 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.911217    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:43:25.911286 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.911217    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:43:25.911675 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.911639    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:43:25.911675 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:25.911633    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-kube-api-access-qpbbn" (OuterVolumeSpecName: "kube-api-access-qpbbn") pod "6d26c5f2-79bc-493a-bf63-13035d7a22c9" (UID: "6d26c5f2-79bc-493a-bf63-13035d7a22c9"). InnerVolumeSpecName "kube-api-access-qpbbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:43:26.010235 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.010209    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:26.010235 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.010231    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:26.010373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.010241    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpbbn\" (UniqueName: \"kubernetes.io/projected/6d26c5f2-79bc-493a-bf63-13035d7a22c9-kube-api-access-qpbbn\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:26.010373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.010251    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:26.010373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.010260    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:26.010373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.010267    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6d26c5f2-79bc-493a-bf63-13035d7a22c9-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:43:26.096274 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.096246    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"]
Jul 06 19:43:26.099732 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.099710    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-dd3bb9-openshift-default-6ff9c6b497-sw9zh"]
Jul 06 19:43:26.928105 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:26.928064    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" path="/var/lib/kubelet/pods/6d26c5f2-79bc-493a-bf63-13035d7a22c9/volumes"
Jul 06 19:43:29.904643 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.904615    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"]
Jul 06 19:43:29.905012 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.904968    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy"
Jul 06 19:43:29.905012 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.904980    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy"
Jul 06 19:43:29.905012 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.904991    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy"
Jul 06 19:43:29.905012 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.904997    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy"
Jul 06 19:43:29.905140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.905048    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy"
Jul 06 19:43:29.905140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.905057    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d26c5f2-79bc-493a-bf63-13035d7a22c9" containerName="istio-proxy"
Jul 06 19:43:29.905140 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.905062    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fcfc4d8-9117-46cd-ab13-f6c25db8da3f" containerName="istio-proxy"
Jul 06 19:43:29.909294 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.909276    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:29.923867 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:29.923844    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"]
Jul 06 19:43:30.041592 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.041557    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-podinfo\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.041592 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.041592    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-token\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.041822 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.041632    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istiod-ca-cert\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.041883 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.041856    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-certs\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.041941 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.041915    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-socket\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.041997 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.041984    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-credential-socket\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.042054 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.042044    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-envoy\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.042148 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.042121    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w4b8\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-kube-api-access-5w4b8\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.042202 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.042187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-data\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.143490 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143461    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-credential-socket\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.143681 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143500    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-envoy\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.143681 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143521    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w4b8\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-kube-api-access-5w4b8\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.143681 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143557    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-data\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.143681 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143600    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-podinfo\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.143681 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143631    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-token\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.143681 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istiod-ca-cert\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.144066 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.143932    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-credential-socket\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.144128 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.144065    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-data\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.144250 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.144224    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-certs\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.144330 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.144284    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-socket\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.144394 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.144351    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istiod-ca-cert\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.144530 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.144509    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-certs\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.144592 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.144553    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-socket\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.145857 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.145835    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-envoy\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.146205 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.146187    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-podinfo\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.152982 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.152949    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w4b8\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-kube-api-access-5w4b8\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.153150 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.153129    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-token\") pod \"e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") " pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.218828 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.218764    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:30.360328 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.360305    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"]
Jul 06 19:43:30.362092 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:43:30.362067    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e38678c_7b2c_494a_83ee_9abbc4f4f33d.slice/crio-1d2468be73af467ef55754be591093c8dd18809820c106f5e33c35dc091b9238 WatchSource:0}: Error finding container 1d2468be73af467ef55754be591093c8dd18809820c106f5e33c35dc091b9238: Status 404 returned error can't find the container with id 1d2468be73af467ef55754be591093c8dd18809820c106f5e33c35dc091b9238
Jul 06 19:43:30.364254 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.364224    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:30.364329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.364288    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:30.364329 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.364320    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:30.796105 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.796070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" event={"ID":"6e38678c-7b2c-494a-83ee-9abbc4f4f33d","Type":"ContainerStarted","Data":"e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16"}
Jul 06 19:43:30.796248 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.796121    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" event={"ID":"6e38678c-7b2c-494a-83ee-9abbc4f4f33d","Type":"ContainerStarted","Data":"1d2468be73af467ef55754be591093c8dd18809820c106f5e33c35dc091b9238"}
Jul 06 19:43:30.818903 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:30.818852    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" podStartSLOduration=1.8188333719999998 podStartE2EDuration="1.818833372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:43:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:43:30.817680055 +0000 UTC m=+2670.429005592" watchObservedRunningTime="2026-07-06 19:43:30.818833372 +0000 UTC m=+2670.430158913"
Jul 06 19:43:31.219875 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:31.219792    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:31.224896 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:31.224872    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:31.801031 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:31.801001    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:31.801850 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:31.801831    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:40.292618 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.291300    2579 patch_prober.go:28] interesting pod/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:48040->10.133.0.57:15021: read: connection reset by peer" start-of-body=
Jul 06 19:43:40.292618 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.291358    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:48040->10.133.0.57:15021: read: connection reset by peer"
Jul 06 19:43:40.860545 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.860507    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerID="b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e" exitCode=0
Jul 06 19:43:40.860743 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.860578    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" event={"ID":"7e4c86e6-3a64-4227-89e4-da6961b83b50","Type":"ContainerDied","Data":"b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e"}
Jul 06 19:43:40.860990 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.860972    2579 scope.go:117] "RemoveContainer" containerID="b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e"
Jul 06 19:43:40.862000 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.861974    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:40.862119 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.862058    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:40.862119 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:40.862089    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:41.871430 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:41.871388    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" event={"ID":"7e4c86e6-3a64-4227-89e4-da6961b83b50","Type":"ContainerStarted","Data":"dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5"}
Jul 06 19:43:42.812807 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:42.812769    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:42.812983 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:42.812937    2579 patch_prober.go:28] interesting pod/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 06 19:43:42.813029 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:42.812981    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 06 19:43:43.813933 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:43.813900    2579 patch_prober.go:28] interesting pod/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 06 19:43:43.814297 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:43.813953    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 06 19:43:44.813842 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:44.813802    2579 patch_prober.go:28] interesting pod/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Jul 06 19:43:44.814045 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:44.813861    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Jul 06 19:43:45.817310 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:45.817276    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:45.817801 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:45.817685    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:45.818362 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:45.818342    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:43:46.802430 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.802357    2579 patch_prober.go:28] interesting pod/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 19:43:46.802620 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.802420    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 19:43:46.901317 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.901247    2579 generic.go:356] "Generic (PLEG): container finished" podID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerID="e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16" exitCode=0
Jul 06 19:43:46.901637 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.901323    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" event={"ID":"6e38678c-7b2c-494a-83ee-9abbc4f4f33d","Type":"ContainerDied","Data":"e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16"}
Jul 06 19:43:46.901816 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.901800    2579 scope.go:117] "RemoveContainer" containerID="e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16"
Jul 06 19:43:46.902909 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.902882    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:46.902995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.902947    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:46.902995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:46.902976    2579 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 06 19:43:47.910508 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:47.910471    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" event={"ID":"6e38678c-7b2c-494a-83ee-9abbc4f4f33d","Type":"ContainerStarted","Data":"140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c"}
Jul 06 19:43:48.219516 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:48.219434    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:48.219682 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:48.219641    2579 patch_prober.go:28] interesting pod/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 19:43:48.219737 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:48.219715    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 19:43:49.219340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:49.219312    2579 patch_prober.go:28] interesting pod/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 19:43:49.219722 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:49.219364    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 19:43:50.219836 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:50.219806    2579 patch_prober.go:28] interesting pod/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 19:43:50.220208 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:50.219856    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 19:43:51.219943 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:51.219914    2579 patch_prober.go:28] interesting pod/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body=
Jul 06 19:43:51.220294 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:51.219959    2579 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused"
Jul 06 19:43:52.222753 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:52.222723    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:52.223129 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:52.223081    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:43:52.223745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:43:52.223726    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:44:06.547761 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.547728    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"]
Jul 06 19:44:06.552444 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.552428    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.558706 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.558688    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"kube-root-ca.crt\""
Jul 06 19:44:06.559043 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.559027    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"openshift-service-ca.crt\""
Jul 06 19:44:06.559843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.559823    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"auth-test-model-5c92ad-kserve-self-signed-certs\""
Jul 06 19:44:06.588771 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.588747    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"]
Jul 06 19:44:06.653301 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.653279    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hv9ps\" (UniqueName: \"kubernetes.io/projected/03f4b9f2-73f1-4ca3-b491-1abbf442b864-kube-api-access-hv9ps\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.653408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.653323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-dshm\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.653408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.653377    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tls-certs\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.653482 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.653423    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-home\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.653482 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.653440    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-model-cache\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.653482 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.653454    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tmp-dir\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.754458 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754423    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-dshm\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.754604 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754478    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tls-certs\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.754604 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754520    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-home\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.754604 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754541    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-model-cache\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.754604 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754564    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tmp-dir\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.754889 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754632    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hv9ps\" (UniqueName: \"kubernetes.io/projected/03f4b9f2-73f1-4ca3-b491-1abbf442b864-kube-api-access-hv9ps\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.754943 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754926    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-model-cache\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.755018 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.754996    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tmp-dir\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.755063 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.755004    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-home\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.756728 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.756709    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-dshm\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.756884 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.756858    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tls-certs\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.764107 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.764087    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hv9ps\" (UniqueName: \"kubernetes.io/projected/03f4b9f2-73f1-4ca3-b491-1abbf442b864-kube-api-access-hv9ps\") pod \"auth-test-model-5c92ad-kserve-676cd4b67-vps26\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") " pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:06.860962 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:06.860904    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:07.000738 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:07.000714    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"]
Jul 06 19:44:07.002774 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:44:07.002735    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod03f4b9f2_73f1_4ca3_b491_1abbf442b864.slice/crio-3314494c0a4b99823b163082b2a45817601339f484e27af970f45cee44ef6748 WatchSource:0}: Error finding container 3314494c0a4b99823b163082b2a45817601339f484e27af970f45cee44ef6748: Status 404 returned error can't find the container with id 3314494c0a4b99823b163082b2a45817601339f484e27af970f45cee44ef6748
Jul 06 19:44:08.013264 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:08.013224    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26" event={"ID":"03f4b9f2-73f1-4ca3-b491-1abbf442b864","Type":"ContainerStarted","Data":"4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512"}
Jul 06 19:44:08.013632 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:08.013277    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26" event={"ID":"03f4b9f2-73f1-4ca3-b491-1abbf442b864","Type":"ContainerStarted","Data":"3314494c0a4b99823b163082b2a45817601339f484e27af970f45cee44ef6748"}
Jul 06 19:44:08.041031 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:08.040984    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26" podStartSLOduration=2.040970607 podStartE2EDuration="2.040970607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:44:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:44:08.039096137 +0000 UTC m=+2707.650421675" watchObservedRunningTime="2026-07-06 19:44:08.040970607 +0000 UTC m=+2707.652296162"
Jul 06 19:44:16.861504 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:16.861474    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:16.861979 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:16.861517    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:16.874105 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:16.874081    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:44:17.070171 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:44:17.070142    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:45:00.147997 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.147949    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"]
Jul 06 19:45:00.151870 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.151852    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722785-ktdkt"]
Jul 06 19:45:00.152002 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.151977    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"
Jul 06 19:45:00.154718 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.154697    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-zxgdj\""
Jul 06 19:45:00.155046 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.155030    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt"
Jul 06 19:45:00.162911 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.162888    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"]
Jul 06 19:45:00.167561 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.167537    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722785-ktdkt"]
Jul 06 19:45:00.194982 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.194950    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tbbk\" (UniqueName: \"kubernetes.io/projected/806adc84-fd44-4c52-ac87-47e55e9b9fda-kube-api-access-4tbbk\") pod \"maas-api-key-cleanup-29722785-ktdkt\" (UID: \"806adc84-fd44-4c52-ac87-47e55e9b9fda\") " pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt"
Jul 06 19:45:00.195112 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.195061    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k585j\" (UniqueName: \"kubernetes.io/projected/96403e99-c032-46f7-80fa-df8bc13a28c1-kube-api-access-k585j\") pod \"maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t\" (UID: \"96403e99-c032-46f7-80fa-df8bc13a28c1\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"
Jul 06 19:45:00.296249 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.296222    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tbbk\" (UniqueName: \"kubernetes.io/projected/806adc84-fd44-4c52-ac87-47e55e9b9fda-kube-api-access-4tbbk\") pod \"maas-api-key-cleanup-29722785-ktdkt\" (UID: \"806adc84-fd44-4c52-ac87-47e55e9b9fda\") " pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt"
Jul 06 19:45:00.296390 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.296272    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k585j\" (UniqueName: \"kubernetes.io/projected/96403e99-c032-46f7-80fa-df8bc13a28c1-kube-api-access-k585j\") pod \"maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t\" (UID: \"96403e99-c032-46f7-80fa-df8bc13a28c1\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"
Jul 06 19:45:00.304989 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.304961    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k585j\" (UniqueName: \"kubernetes.io/projected/96403e99-c032-46f7-80fa-df8bc13a28c1-kube-api-access-k585j\") pod \"maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t\" (UID: \"96403e99-c032-46f7-80fa-df8bc13a28c1\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"
Jul 06 19:45:00.305096 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.305068    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tbbk\" (UniqueName: \"kubernetes.io/projected/806adc84-fd44-4c52-ac87-47e55e9b9fda-kube-api-access-4tbbk\") pod \"maas-api-key-cleanup-29722785-ktdkt\" (UID: \"806adc84-fd44-4c52-ac87-47e55e9b9fda\") " pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt"
Jul 06 19:45:00.465889 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.465821    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"
Jul 06 19:45:00.476519 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.476495    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt"
Jul 06 19:45:00.609551 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.609508    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"]
Jul 06 19:45:00.610542 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:45:00.610511    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96403e99_c032_46f7_80fa_df8bc13a28c1.slice/crio-68cfb26e6b824a28b36777e17f0bdd5055d1bec269fe3a78dfe59d15eb059c2c WatchSource:0}: Error finding container 68cfb26e6b824a28b36777e17f0bdd5055d1bec269fe3a78dfe59d15eb059c2c: Status 404 returned error can't find the container with id 68cfb26e6b824a28b36777e17f0bdd5055d1bec269fe3a78dfe59d15eb059c2c
Jul 06 19:45:00.634069 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:00.634050    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722785-ktdkt"]
Jul 06 19:45:00.640630 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:45:00.640604    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806adc84_fd44_4c52_ac87_47e55e9b9fda.slice/crio-81a8fecbbaf7f7293024ceaa0eb13753e71b91be4594550673cd71d5b159006e WatchSource:0}: Error finding container 81a8fecbbaf7f7293024ceaa0eb13753e71b91be4594550673cd71d5b159006e: Status 404 returned error can't find the container with id 81a8fecbbaf7f7293024ceaa0eb13753e71b91be4594550673cd71d5b159006e
Jul 06 19:45:01.282261 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:01.282168    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t" event={"ID":"96403e99-c032-46f7-80fa-df8bc13a28c1","Type":"ContainerStarted","Data":"f32c285e9ca073698a2f2b8e1c65ffb1c821130a1165d63a9e313e7529c070bf"}
Jul 06 19:45:01.282261 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:01.282231    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t" event={"ID":"96403e99-c032-46f7-80fa-df8bc13a28c1","Type":"ContainerStarted","Data":"68cfb26e6b824a28b36777e17f0bdd5055d1bec269fe3a78dfe59d15eb059c2c"}
Jul 06 19:45:01.283619 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:01.283595    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt" event={"ID":"806adc84-fd44-4c52-ac87-47e55e9b9fda","Type":"ContainerStarted","Data":"12bdcdf682346de6493eae3f3cb16fd3aae8549e9644a13028173119e71fe345"}
Jul 06 19:45:01.283775 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:01.283635    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt" event={"ID":"806adc84-fd44-4c52-ac87-47e55e9b9fda","Type":"ContainerStarted","Data":"81a8fecbbaf7f7293024ceaa0eb13753e71b91be4594550673cd71d5b159006e"}
Jul 06 19:45:01.306285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:01.306229    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t" podStartSLOduration=1.306211736 podStartE2EDuration="1.306211736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:45:01.303937678 +0000 UTC m=+2760.915263216" watchObservedRunningTime="2026-07-06 19:45:01.306211736 +0000 UTC m=+2760.917537275"
Jul 06 19:45:01.321779 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:01.321738    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt" podStartSLOduration=1.32172593 podStartE2EDuration="1.32172593s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:45:01.319465381 +0000 UTC m=+2760.930790931" watchObservedRunningTime="2026-07-06 19:45:01.32172593 +0000 UTC m=+2760.933051489"
Jul 06 19:45:02.290550 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:02.290510    2579 generic.go:356] "Generic (PLEG): container finished" podID="806adc84-fd44-4c52-ac87-47e55e9b9fda" containerID="12bdcdf682346de6493eae3f3cb16fd3aae8549e9644a13028173119e71fe345" exitCode=0
Jul 06 19:45:02.291029 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:02.290602    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt" event={"ID":"806adc84-fd44-4c52-ac87-47e55e9b9fda","Type":"ContainerDied","Data":"12bdcdf682346de6493eae3f3cb16fd3aae8549e9644a13028173119e71fe345"}
Jul 06 19:45:02.292138 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:02.292117    2579 generic.go:356] "Generic (PLEG): container finished" podID="96403e99-c032-46f7-80fa-df8bc13a28c1" containerID="f32c285e9ca073698a2f2b8e1c65ffb1c821130a1165d63a9e313e7529c070bf" exitCode=0
Jul 06 19:45:02.292233 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:02.292154    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t" event={"ID":"96403e99-c032-46f7-80fa-df8bc13a28c1","Type":"ContainerDied","Data":"f32c285e9ca073698a2f2b8e1c65ffb1c821130a1165d63a9e313e7529c070bf"}
Jul 06 19:45:03.444777 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.444751    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt"
Jul 06 19:45:03.482485 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.482452    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"
Jul 06 19:45:03.524666 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.524627    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k585j\" (UniqueName: \"kubernetes.io/projected/96403e99-c032-46f7-80fa-df8bc13a28c1-kube-api-access-k585j\") pod \"96403e99-c032-46f7-80fa-df8bc13a28c1\" (UID: \"96403e99-c032-46f7-80fa-df8bc13a28c1\") "
Jul 06 19:45:03.524818 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.524693    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tbbk\" (UniqueName: \"kubernetes.io/projected/806adc84-fd44-4c52-ac87-47e55e9b9fda-kube-api-access-4tbbk\") pod \"806adc84-fd44-4c52-ac87-47e55e9b9fda\" (UID: \"806adc84-fd44-4c52-ac87-47e55e9b9fda\") "
Jul 06 19:45:03.526799 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.526774    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96403e99-c032-46f7-80fa-df8bc13a28c1-kube-api-access-k585j" (OuterVolumeSpecName: "kube-api-access-k585j") pod "96403e99-c032-46f7-80fa-df8bc13a28c1" (UID: "96403e99-c032-46f7-80fa-df8bc13a28c1"). InnerVolumeSpecName "kube-api-access-k585j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:45:03.526910 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.526840    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/806adc84-fd44-4c52-ac87-47e55e9b9fda-kube-api-access-4tbbk" (OuterVolumeSpecName: "kube-api-access-4tbbk") pod "806adc84-fd44-4c52-ac87-47e55e9b9fda" (UID: "806adc84-fd44-4c52-ac87-47e55e9b9fda"). InnerVolumeSpecName "kube-api-access-4tbbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:45:03.625844 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.625761    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k585j\" (UniqueName: \"kubernetes.io/projected/96403e99-c032-46f7-80fa-df8bc13a28c1-kube-api-access-k585j\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:45:03.625844 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:03.625791    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tbbk\" (UniqueName: \"kubernetes.io/projected/806adc84-fd44-4c52-ac87-47e55e9b9fda-kube-api-access-4tbbk\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:45:04.306673 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:04.306631    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt"
Jul 06 19:45:04.306855 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:04.306688    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722785-ktdkt" event={"ID":"806adc84-fd44-4c52-ac87-47e55e9b9fda","Type":"ContainerDied","Data":"81a8fecbbaf7f7293024ceaa0eb13753e71b91be4594550673cd71d5b159006e"}
Jul 06 19:45:04.306855 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:04.306736    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81a8fecbbaf7f7293024ceaa0eb13753e71b91be4594550673cd71d5b159006e"
Jul 06 19:45:04.307946 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:04.307918    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t" event={"ID":"96403e99-c032-46f7-80fa-df8bc13a28c1","Type":"ContainerDied","Data":"68cfb26e6b824a28b36777e17f0bdd5055d1bec269fe3a78dfe59d15eb059c2c"}
Jul 06 19:45:04.307946 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:04.307929    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"
Jul 06 19:45:04.308099 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:45:04.307954    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68cfb26e6b824a28b36777e17f0bdd5055d1bec269fe3a78dfe59d15eb059c2c"
Jul 06 19:46:46.181143 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:46:46.181106    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"]
Jul 06 19:47:01.200723 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.200669    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26" podUID="03f4b9f2-73f1-4ca3-b491-1abbf442b864" containerName="main" containerID="cri-o://4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512" gracePeriod=45
Jul 06 19:47:01.444885 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.444863    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:47:01.507547 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507483    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tmp-dir\") pod \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") "
Jul 06 19:47:01.507547 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507520    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hv9ps\" (UniqueName: \"kubernetes.io/projected/03f4b9f2-73f1-4ca3-b491-1abbf442b864-kube-api-access-hv9ps\") pod \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") "
Jul 06 19:47:01.507779 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507548    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-model-cache\") pod \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") "
Jul 06 19:47:01.507779 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507592    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-dshm\") pod \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") "
Jul 06 19:47:01.507779 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507649    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tls-certs\") pod \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") "
Jul 06 19:47:01.507779 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507725    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-home\") pod \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\" (UID: \"03f4b9f2-73f1-4ca3-b491-1abbf442b864\") "
Jul 06 19:47:01.507997 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507812    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "03f4b9f2-73f1-4ca3-b491-1abbf442b864" (UID: "03f4b9f2-73f1-4ca3-b491-1abbf442b864"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:47:01.507997 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507891    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-model-cache" (OuterVolumeSpecName: "model-cache") pod "03f4b9f2-73f1-4ca3-b491-1abbf442b864" (UID: "03f4b9f2-73f1-4ca3-b491-1abbf442b864"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:47:01.507997 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.507977    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-home" (OuterVolumeSpecName: "home") pod "03f4b9f2-73f1-4ca3-b491-1abbf442b864" (UID: "03f4b9f2-73f1-4ca3-b491-1abbf442b864"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:47:01.508158 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.508066    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:47:01.508158 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.508082    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:47:01.509823 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.509796    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-dshm" (OuterVolumeSpecName: "dshm") pod "03f4b9f2-73f1-4ca3-b491-1abbf442b864" (UID: "03f4b9f2-73f1-4ca3-b491-1abbf442b864"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:47:01.509919 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.509836    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "03f4b9f2-73f1-4ca3-b491-1abbf442b864" (UID: "03f4b9f2-73f1-4ca3-b491-1abbf442b864"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:47:01.509919 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.509861    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03f4b9f2-73f1-4ca3-b491-1abbf442b864-kube-api-access-hv9ps" (OuterVolumeSpecName: "kube-api-access-hv9ps") pod "03f4b9f2-73f1-4ca3-b491-1abbf442b864" (UID: "03f4b9f2-73f1-4ca3-b491-1abbf442b864"). InnerVolumeSpecName "kube-api-access-hv9ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:47:01.609172 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.609132    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:47:01.609309 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.609182    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03f4b9f2-73f1-4ca3-b491-1abbf442b864-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:47:01.609309 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.609194    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03f4b9f2-73f1-4ca3-b491-1abbf442b864-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:47:01.609309 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.609203    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hv9ps\" (UniqueName: \"kubernetes.io/projected/03f4b9f2-73f1-4ca3-b491-1abbf442b864-kube-api-access-hv9ps\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:47:01.907259 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.907228    2579 generic.go:356] "Generic (PLEG): container finished" podID="03f4b9f2-73f1-4ca3-b491-1abbf442b864" containerID="4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512" exitCode=0
Jul 06 19:47:01.907427 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.907269    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"
Jul 06 19:47:01.907427 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.907268    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26" event={"ID":"03f4b9f2-73f1-4ca3-b491-1abbf442b864","Type":"ContainerDied","Data":"4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512"}
Jul 06 19:47:01.907427 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.907406    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26" event={"ID":"03f4b9f2-73f1-4ca3-b491-1abbf442b864","Type":"ContainerDied","Data":"3314494c0a4b99823b163082b2a45817601339f484e27af970f45cee44ef6748"}
Jul 06 19:47:01.907600 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.907439    2579 scope.go:117] "RemoveContainer" containerID="4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512"
Jul 06 19:47:01.917227 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.917212    2579 scope.go:117] "RemoveContainer" containerID="4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512"
Jul 06 19:47:01.917477 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:47:01.917457    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512\": container with ID starting with 4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512 not found: ID does not exist" containerID="4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512"
Jul 06 19:47:01.917569 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.917481    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512"} err="failed to get container status \"4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512\": rpc error: code = NotFound desc = could not find container \"4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512\": container with ID starting with 4974c8b7bfbd41639064113c0c8e8c0f7c0a5405b7256f713299ac561bb52512 not found: ID does not exist"
Jul 06 19:47:01.929456 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.929433    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"]
Jul 06 19:47:01.933948 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:01.933927    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/auth-test-model-5c92ad-kserve-676cd4b67-vps26"]
Jul 06 19:47:02.927111 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:02.927075    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03f4b9f2-73f1-4ca3-b491-1abbf442b864" path="/var/lib/kubelet/pods/03f4b9f2-73f1-4ca3-b491-1abbf442b864/volumes"
Jul 06 19:47:07.088745 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.088709    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"]
Jul 06 19:47:07.089206 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089181    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03f4b9f2-73f1-4ca3-b491-1abbf442b864" containerName="main"
Jul 06 19:47:07.089206 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089200    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="03f4b9f2-73f1-4ca3-b491-1abbf442b864" containerName="main"
Jul 06 19:47:07.089333 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089229    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96403e99-c032-46f7-80fa-df8bc13a28c1" containerName="cleanup"
Jul 06 19:47:07.089333 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089239    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="96403e99-c032-46f7-80fa-df8bc13a28c1" containerName="cleanup"
Jul 06 19:47:07.089333 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089254    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="806adc84-fd44-4c52-ac87-47e55e9b9fda" containerName="cleanup"
Jul 06 19:47:07.089333 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089263    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="806adc84-fd44-4c52-ac87-47e55e9b9fda" containerName="cleanup"
Jul 06 19:47:07.089528 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089364    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="03f4b9f2-73f1-4ca3-b491-1abbf442b864" containerName="main"
Jul 06 19:47:07.089528 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089380    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="806adc84-fd44-4c52-ac87-47e55e9b9fda" containerName="cleanup"
Jul 06 19:47:07.089528 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.089392    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="96403e99-c032-46f7-80fa-df8bc13a28c1" containerName="cleanup"
Jul 06 19:47:07.093900 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.093879    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.106760 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.106737    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"sub-test-model-5c92ad-kserve-self-signed-certs\""
Jul 06 19:47:07.106865 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.106741    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"openshift-service-ca.crt\""
Jul 06 19:47:07.106865 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.106771    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"default-dockercfg-f5zm5\""
Jul 06 19:47:07.107339 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.107323    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"kube-root-ca.crt\""
Jul 06 19:47:07.116767 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.116744    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"]
Jul 06 19:47:07.155287 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.155264    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqwzr\" (UniqueName: \"kubernetes.io/projected/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-kube-api-access-nqwzr\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.155397 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.155299    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-model-cache\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.155397 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.155323    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tmp-dir\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.155475 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.155448    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-home\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.155511 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.155472    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tls-certs\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.155511 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.155496    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-dshm\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.256754 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.256718    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-home\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.256754 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.256753    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tls-certs\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.257011 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.256776    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-dshm\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.257011 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.256821    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqwzr\" (UniqueName: \"kubernetes.io/projected/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-kube-api-access-nqwzr\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.257011 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.256844    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-model-cache\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.257011 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.256863    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tmp-dir\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.257313 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.257289    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tmp-dir\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.257313 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.257303    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-home\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.257433 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.257397    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-model-cache\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.259388 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.259341    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-dshm\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.259520 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.259479    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tls-certs\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.266286 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.266258    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqwzr\" (UniqueName: \"kubernetes.io/projected/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-kube-api-access-nqwzr\") pod \"sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") " pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.402511 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.402442    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:07.537805 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.537781    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"]
Jul 06 19:47:07.540035 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:47:07.539998    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc09785f_0bd2_4604_aaad_d6ca599a6b0b.slice/crio-79564fa6316c1b03fe9df06fb08ac5559133bafe38b1883c0629546710cc830b WatchSource:0}: Error finding container 79564fa6316c1b03fe9df06fb08ac5559133bafe38b1883c0629546710cc830b: Status 404 returned error can't find the container with id 79564fa6316c1b03fe9df06fb08ac5559133bafe38b1883c0629546710cc830b
Jul 06 19:47:07.941563 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.941526    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq" event={"ID":"fc09785f-0bd2-4604-aaad-d6ca599a6b0b","Type":"ContainerStarted","Data":"8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587"}
Jul 06 19:47:07.941751 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.941571    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq" event={"ID":"fc09785f-0bd2-4604-aaad-d6ca599a6b0b","Type":"ContainerStarted","Data":"79564fa6316c1b03fe9df06fb08ac5559133bafe38b1883c0629546710cc830b"}
Jul 06 19:47:07.962227 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:07.962176    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq" podStartSLOduration=0.962159718 podStartE2EDuration="962.159718ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:47:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:47:07.960950318 +0000 UTC m=+2887.572275856" watchObservedRunningTime="2026-07-06 19:47:07.962159718 +0000 UTC m=+2887.573485255"
Jul 06 19:47:17.403584 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:17.403546    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:17.404127 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:17.403862    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:17.416439 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:17.416413    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:47:18.003536 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:47:18.003511    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:48:22.899968 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:22.899897    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"]
Jul 06 19:48:37.919030 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:37.918969    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq" podUID="fc09785f-0bd2-4604-aaad-d6ca599a6b0b" containerName="main" containerID="cri-o://8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587" gracePeriod=45
Jul 06 19:48:37.992999 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:37.992966    2579 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq" podUID="fc09785f-0bd2-4604-aaad-d6ca599a6b0b" containerName="main" probeResult="failure" output="Get \"https://10.133.0.62:8000/ready\": dial tcp 10.133.0.62:8000: connect: connection refused"
Jul 06 19:48:38.166555 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.166531    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:48:38.309373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309348    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tls-certs\") pod \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") "
Jul 06 19:48:38.309531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309396    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-model-cache\") pod \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") "
Jul 06 19:48:38.309531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309433    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-dshm\") pod \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") "
Jul 06 19:48:38.309531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309466    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-home\") pod \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") "
Jul 06 19:48:38.309531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309522    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqwzr\" (UniqueName: \"kubernetes.io/projected/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-kube-api-access-nqwzr\") pod \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") "
Jul 06 19:48:38.309781 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309563    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tmp-dir\") pod \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\" (UID: \"fc09785f-0bd2-4604-aaad-d6ca599a6b0b\") "
Jul 06 19:48:38.309838 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309769    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-home" (OuterVolumeSpecName: "home") pod "fc09785f-0bd2-4604-aaad-d6ca599a6b0b" (UID: "fc09785f-0bd2-4604-aaad-d6ca599a6b0b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:48:38.309838 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309779    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-model-cache" (OuterVolumeSpecName: "model-cache") pod "fc09785f-0bd2-4604-aaad-d6ca599a6b0b" (UID: "fc09785f-0bd2-4604-aaad-d6ca599a6b0b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:48:38.309918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309872    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fc09785f-0bd2-4604-aaad-d6ca599a6b0b" (UID: "fc09785f-0bd2-4604-aaad-d6ca599a6b0b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:48:38.309918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309898    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:48:38.309918 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.309916    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:48:38.311532 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.311496    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-kube-api-access-nqwzr" (OuterVolumeSpecName: "kube-api-access-nqwzr") pod "fc09785f-0bd2-4604-aaad-d6ca599a6b0b" (UID: "fc09785f-0bd2-4604-aaad-d6ca599a6b0b"). InnerVolumeSpecName "kube-api-access-nqwzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:48:38.311532 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.311508    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-dshm" (OuterVolumeSpecName: "dshm") pod "fc09785f-0bd2-4604-aaad-d6ca599a6b0b" (UID: "fc09785f-0bd2-4604-aaad-d6ca599a6b0b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:48:38.311700 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.311566    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fc09785f-0bd2-4604-aaad-d6ca599a6b0b" (UID: "fc09785f-0bd2-4604-aaad-d6ca599a6b0b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:48:38.396215 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.396187    2579 generic.go:356] "Generic (PLEG): container finished" podID="fc09785f-0bd2-4604-aaad-d6ca599a6b0b" containerID="8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587" exitCode=0
Jul 06 19:48:38.396356 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.396229    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"
Jul 06 19:48:38.396356 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.396239    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq" event={"ID":"fc09785f-0bd2-4604-aaad-d6ca599a6b0b","Type":"ContainerDied","Data":"8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587"}
Jul 06 19:48:38.396356 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.396273    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq" event={"ID":"fc09785f-0bd2-4604-aaad-d6ca599a6b0b","Type":"ContainerDied","Data":"79564fa6316c1b03fe9df06fb08ac5559133bafe38b1883c0629546710cc830b"}
Jul 06 19:48:38.396356 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.396296    2579 scope.go:117] "RemoveContainer" containerID="8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587"
Jul 06 19:48:38.405549 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.405532    2579 scope.go:117] "RemoveContainer" containerID="8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587"
Jul 06 19:48:38.405847 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:48:38.405829    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587\": container with ID starting with 8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587 not found: ID does not exist" containerID="8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587"
Jul 06 19:48:38.405914 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.405856    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587"} err="failed to get container status \"8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587\": rpc error: code = NotFound desc = could not find container \"8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587\": container with ID starting with 8c030ccf7c3813e13ffc326b2187485bf7aa1c743647ee88e2337940b7c9d587 not found: ID does not exist"
Jul 06 19:48:38.410590 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.410570    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:48:38.410672 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.410592    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqwzr\" (UniqueName: \"kubernetes.io/projected/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-kube-api-access-nqwzr\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:48:38.410672 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.410603    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:48:38.410672 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.410613    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fc09785f-0bd2-4604-aaad-d6ca599a6b0b-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:48:38.422480 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.422449    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"]
Jul 06 19:48:38.427524 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.427504    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/sub-test-model-5c92ad-kserve-6d55dc85ff-s9bvq"]
Jul 06 19:48:38.927991 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:38.927962    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc09785f-0bd2-4604-aaad-d6ca599a6b0b" path="/var/lib/kubelet/pods/fc09785f-0bd2-4604-aaad-d6ca599a6b0b/volumes"
Jul 06 19:48:40.385651 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.385611    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"]
Jul 06 19:48:40.386132 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.386110    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc09785f-0bd2-4604-aaad-d6ca599a6b0b" containerName="main"
Jul 06 19:48:40.386207 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.386137    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc09785f-0bd2-4604-aaad-d6ca599a6b0b" containerName="main"
Jul 06 19:48:40.386267 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.386246    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc09785f-0bd2-4604-aaad-d6ca599a6b0b" containerName="main"
Jul 06 19:48:40.390596 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.390577    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.393671 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.393642    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"kube-root-ca.crt\""
Jul 06 19:48:40.394772 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.394754    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"rate-test-model-5c92ad-kserve-self-signed-certs\""
Jul 06 19:48:40.395024 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.394993    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"default-dockercfg-f5zm5\""
Jul 06 19:48:40.395126 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.395072    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-5c92ad\"/\"openshift-service-ca.crt\""
Jul 06 19:48:40.407272 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.407252    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"]
Jul 06 19:48:40.528962 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.528933    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-tmp-dir\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.529102 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.528967    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-home\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.529102 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.528986    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9gkwk\" (UniqueName: \"kubernetes.io/projected/d7c92d8f-429b-4744-928b-a45c843d0d82-kube-api-access-9gkwk\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.529102 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.529087    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-dshm\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.529210 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.529118    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-model-cache\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.529210 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.529156    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7c92d8f-429b-4744-928b-a45c843d0d82-tls-certs\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630284 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630251    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-tmp-dir\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630284 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630287    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-home\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630311    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9gkwk\" (UniqueName: \"kubernetes.io/projected/d7c92d8f-429b-4744-928b-a45c843d0d82-kube-api-access-9gkwk\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630371    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-dshm\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630395    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-model-cache\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630431    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7c92d8f-429b-4744-928b-a45c843d0d82-tls-certs\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630744 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630632    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-tmp-dir\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630744 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630733    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-home\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.630862 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.630843    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-model-cache\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.632563 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.632541    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-dshm\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.632958 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.632939    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7c92d8f-429b-4744-928b-a45c843d0d82-tls-certs\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.643866 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.643812    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9gkwk\" (UniqueName: \"kubernetes.io/projected/d7c92d8f-429b-4744-928b-a45c843d0d82-kube-api-access-9gkwk\") pod \"rate-test-model-5c92ad-kserve-7b4b474675-d49hq\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") " pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.699833 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.699803    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:40.850432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:40.850404    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"]
Jul 06 19:48:40.851981 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:48:40.851954    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7c92d8f_429b_4744_928b_a45c843d0d82.slice/crio-7cb20a6ea6e46b3a84f50daaadad8bfdf157afbc7cf1ca8b1fe1b72c9be03d9d WatchSource:0}: Error finding container 7cb20a6ea6e46b3a84f50daaadad8bfdf157afbc7cf1ca8b1fe1b72c9be03d9d: Status 404 returned error can't find the container with id 7cb20a6ea6e46b3a84f50daaadad8bfdf157afbc7cf1ca8b1fe1b72c9be03d9d
Jul 06 19:48:41.415506 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:41.415467    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq" event={"ID":"d7c92d8f-429b-4744-928b-a45c843d0d82","Type":"ContainerStarted","Data":"67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7"}
Jul 06 19:48:41.415892 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:41.415517    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq" event={"ID":"d7c92d8f-429b-4744-928b-a45c843d0d82","Type":"ContainerStarted","Data":"7cb20a6ea6e46b3a84f50daaadad8bfdf157afbc7cf1ca8b1fe1b72c9be03d9d"}
Jul 06 19:48:41.441306 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:41.441258    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq" podStartSLOduration=1.441242736 podStartE2EDuration="1.441242736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:48:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:48:41.439836477 +0000 UTC m=+2981.051162016" watchObservedRunningTime="2026-07-06 19:48:41.441242736 +0000 UTC m=+2981.052568273"
Jul 06 19:48:50.700708 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:50.700674    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:50.703040 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:50.700721    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:50.713548 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:50.713524    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:48:51.477214 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:48:51.477181    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:49:28.839460 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.839427    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"]
Jul 06 19:49:28.843084 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.843061    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:28.846598 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.846573    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5f29b8\"/\"openshift-service-ca.crt\""
Jul 06 19:49:28.847014 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.846999    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-5f29b8\"/\"kube-root-ca.crt\""
Jul 06 19:49:28.848142 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.848121    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5f29b8\"/\"rate-test-model-5f29b8-kserve-self-signed-certs\""
Jul 06 19:49:28.848238 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.848221    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-5f29b8\"/\"default-dockercfg-j9f99\""
Jul 06 19:49:28.857185 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.857155    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"]
Jul 06 19:49:28.939149 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.939093    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-tmp-dir\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:28.939149 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.939140    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwkrb\" (UniqueName: \"kubernetes.io/projected/aeb9a8de-fbeb-4923-8824-2b7044af371a-kube-api-access-pwkrb\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:28.939539 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.939192    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-model-cache\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:28.939539 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.939214    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb9a8de-fbeb-4923-8824-2b7044af371a-tls-certs\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:28.939539 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.939259    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-home\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:28.939539 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:28.939285    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-dshm\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.040710 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.040677    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-model-cache\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.040710 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.040712    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb9a8de-fbeb-4923-8824-2b7044af371a-tls-certs\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.040957 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.040740    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-home\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.040957 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.040757    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-dshm\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.040957 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.040823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-tmp-dir\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.040957 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.040838    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pwkrb\" (UniqueName: \"kubernetes.io/projected/aeb9a8de-fbeb-4923-8824-2b7044af371a-kube-api-access-pwkrb\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.041174 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.041158    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-home\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.041229 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.041182    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-model-cache\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.041282 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.041262    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-tmp-dir\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.043043 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.043018    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-dshm\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.043303 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.043287    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb9a8de-fbeb-4923-8824-2b7044af371a-tls-certs\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.050628 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.050605    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwkrb\" (UniqueName: \"kubernetes.io/projected/aeb9a8de-fbeb-4923-8824-2b7044af371a-kube-api-access-pwkrb\") pod \"rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") " pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.151798 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.151723    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:29.287940 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.287909    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"]
Jul 06 19:49:29.289218 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:49:29.289187    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaeb9a8de_fbeb_4923_8824_2b7044af371a.slice/crio-115bcb3e271d5c54c855282ddf48636f7a787fb69e570431b6b99a6c14f6f57f WatchSource:0}: Error finding container 115bcb3e271d5c54c855282ddf48636f7a787fb69e570431b6b99a6c14f6f57f: Status 404 returned error can't find the container with id 115bcb3e271d5c54c855282ddf48636f7a787fb69e570431b6b99a6c14f6f57f
Jul 06 19:49:29.662937 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.662899    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv" event={"ID":"aeb9a8de-fbeb-4923-8824-2b7044af371a","Type":"ContainerStarted","Data":"82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0"}
Jul 06 19:49:29.663122 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.662948    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv" event={"ID":"aeb9a8de-fbeb-4923-8824-2b7044af371a","Type":"ContainerStarted","Data":"115bcb3e271d5c54c855282ddf48636f7a787fb69e570431b6b99a6c14f6f57f"}
Jul 06 19:49:29.688566 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:29.688516    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv" podStartSLOduration=1.688501037 podStartE2EDuration="1.688501037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:49:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:49:29.687584462 +0000 UTC m=+3029.298909999" watchObservedRunningTime="2026-07-06 19:49:29.688501037 +0000 UTC m=+3029.299826574"
Jul 06 19:49:39.152359 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:39.152323    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:39.152762 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:39.152370    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:39.165125 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:39.165100    2579 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:49:39.726755 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:49:39.726731    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:50:23.041731 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:23.041702    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"]
Jul 06 19:50:23.058373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:23.058346    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"]
Jul 06 19:50:38.061856 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.061795    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq" podUID="d7c92d8f-429b-4744-928b-a45c843d0d82" containerName="main" containerID="cri-o://67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7" gracePeriod=45
Jul 06 19:50:38.077249 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.077199    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv" podUID="aeb9a8de-fbeb-4923-8824-2b7044af371a" containerName="main" containerID="cri-o://82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0" gracePeriod=45
Jul 06 19:50:38.339969 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.339944    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:50:38.342743 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.342724    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:50:38.449173 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449136    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-model-cache\") pod \"d7c92d8f-429b-4744-928b-a45c843d0d82\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") "
Jul 06 19:50:38.449173 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449173    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-dshm\") pod \"aeb9a8de-fbeb-4923-8824-2b7044af371a\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") "
Jul 06 19:50:38.449432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449205    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-tmp-dir\") pod \"d7c92d8f-429b-4744-928b-a45c843d0d82\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") "
Jul 06 19:50:38.449432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449220    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-dshm\") pod \"d7c92d8f-429b-4744-928b-a45c843d0d82\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") "
Jul 06 19:50:38.449432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449236    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwkrb\" (UniqueName: \"kubernetes.io/projected/aeb9a8de-fbeb-4923-8824-2b7044af371a-kube-api-access-pwkrb\") pod \"aeb9a8de-fbeb-4923-8824-2b7044af371a\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") "
Jul 06 19:50:38.449432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449265    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-tmp-dir\") pod \"aeb9a8de-fbeb-4923-8824-2b7044af371a\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") "
Jul 06 19:50:38.449432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449280    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb9a8de-fbeb-4923-8824-2b7044af371a-tls-certs\") pod \"aeb9a8de-fbeb-4923-8824-2b7044af371a\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") "
Jul 06 19:50:38.449432 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449299    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-home\") pod \"aeb9a8de-fbeb-4923-8824-2b7044af371a\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") "
Jul 06 19:50:38.449759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449427    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7c92d8f-429b-4744-928b-a45c843d0d82" (UID: "d7c92d8f-429b-4744-928b-a45c843d0d82"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.449759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449554    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "aeb9a8de-fbeb-4923-8824-2b7044af371a" (UID: "aeb9a8de-fbeb-4923-8824-2b7044af371a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.449759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449553    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7c92d8f-429b-4744-928b-a45c843d0d82" (UID: "d7c92d8f-429b-4744-928b-a45c843d0d82"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.449759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449699    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-home\") pod \"d7c92d8f-429b-4744-928b-a45c843d0d82\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") "
Jul 06 19:50:38.449759 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449746    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9gkwk\" (UniqueName: \"kubernetes.io/projected/d7c92d8f-429b-4744-928b-a45c843d0d82-kube-api-access-9gkwk\") pod \"d7c92d8f-429b-4744-928b-a45c843d0d82\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") "
Jul 06 19:50:38.450016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449782    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7c92d8f-429b-4744-928b-a45c843d0d82-tls-certs\") pod \"d7c92d8f-429b-4744-928b-a45c843d0d82\" (UID: \"d7c92d8f-429b-4744-928b-a45c843d0d82\") "
Jul 06 19:50:38.450016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449824    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-model-cache\") pod \"aeb9a8de-fbeb-4923-8824-2b7044af371a\" (UID: \"aeb9a8de-fbeb-4923-8824-2b7044af371a\") "
Jul 06 19:50:38.450016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.449962    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-home" (OuterVolumeSpecName: "home") pod "d7c92d8f-429b-4744-928b-a45c843d0d82" (UID: "d7c92d8f-429b-4744-928b-a45c843d0d82"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.450172 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.450157    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-model-cache" (OuterVolumeSpecName: "model-cache") pod "aeb9a8de-fbeb-4923-8824-2b7044af371a" (UID: "aeb9a8de-fbeb-4923-8824-2b7044af371a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.450283 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.450235    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.450283 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.450259    2579 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-tmp-dir\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.450400 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.450296    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.450400 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.450309    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.450400 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.450321    2579 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-model-cache\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.450400 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.450337    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-home" (OuterVolumeSpecName: "home") pod "aeb9a8de-fbeb-4923-8824-2b7044af371a" (UID: "aeb9a8de-fbeb-4923-8824-2b7044af371a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.452018 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.451988    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-dshm" (OuterVolumeSpecName: "dshm") pod "aeb9a8de-fbeb-4923-8824-2b7044af371a" (UID: "aeb9a8de-fbeb-4923-8824-2b7044af371a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.452018 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.452011    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-dshm" (OuterVolumeSpecName: "dshm") pod "d7c92d8f-429b-4744-928b-a45c843d0d82" (UID: "d7c92d8f-429b-4744-928b-a45c843d0d82"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:50:38.452215 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.452113    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aeb9a8de-fbeb-4923-8824-2b7044af371a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "aeb9a8de-fbeb-4923-8824-2b7044af371a" (UID: "aeb9a8de-fbeb-4923-8824-2b7044af371a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:50:38.452215 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.452127    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aeb9a8de-fbeb-4923-8824-2b7044af371a-kube-api-access-pwkrb" (OuterVolumeSpecName: "kube-api-access-pwkrb") pod "aeb9a8de-fbeb-4923-8824-2b7044af371a" (UID: "aeb9a8de-fbeb-4923-8824-2b7044af371a"). InnerVolumeSpecName "kube-api-access-pwkrb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:38.452416 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.452395    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7c92d8f-429b-4744-928b-a45c843d0d82-kube-api-access-9gkwk" (OuterVolumeSpecName: "kube-api-access-9gkwk") pod "d7c92d8f-429b-4744-928b-a45c843d0d82" (UID: "d7c92d8f-429b-4744-928b-a45c843d0d82"). InnerVolumeSpecName "kube-api-access-9gkwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:38.452491 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.452449    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7c92d8f-429b-4744-928b-a45c843d0d82-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7c92d8f-429b-4744-928b-a45c843d0d82" (UID: "d7c92d8f-429b-4744-928b-a45c843d0d82"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 19:50:38.551813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.551773    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.551813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.551806    2579 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7c92d8f-429b-4744-928b-a45c843d0d82-dshm\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.551813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.551817    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwkrb\" (UniqueName: \"kubernetes.io/projected/aeb9a8de-fbeb-4923-8824-2b7044af371a-kube-api-access-pwkrb\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.552062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.551827    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/aeb9a8de-fbeb-4923-8824-2b7044af371a-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.552062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.551838    2579 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/aeb9a8de-fbeb-4923-8824-2b7044af371a-home\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.552062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.551846    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9gkwk\" (UniqueName: \"kubernetes.io/projected/d7c92d8f-429b-4744-928b-a45c843d0d82-kube-api-access-9gkwk\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:38.552062 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:38.551853    2579 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7c92d8f-429b-4744-928b-a45c843d0d82-tls-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:39.024229 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.024197    2579 generic.go:356] "Generic (PLEG): container finished" podID="d7c92d8f-429b-4744-928b-a45c843d0d82" containerID="67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7" exitCode=0
Jul 06 19:50:39.024229 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.024234    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"
Jul 06 19:50:39.024467 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.024274    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq" event={"ID":"d7c92d8f-429b-4744-928b-a45c843d0d82","Type":"ContainerDied","Data":"67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7"}
Jul 06 19:50:39.024467 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.024309    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq" event={"ID":"d7c92d8f-429b-4744-928b-a45c843d0d82","Type":"ContainerDied","Data":"7cb20a6ea6e46b3a84f50daaadad8bfdf157afbc7cf1ca8b1fe1b72c9be03d9d"}
Jul 06 19:50:39.024948 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.024328    2579 scope.go:117] "RemoveContainer" containerID="67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7"
Jul 06 19:50:39.025785 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.025742    2579 generic.go:356] "Generic (PLEG): container finished" podID="aeb9a8de-fbeb-4923-8824-2b7044af371a" containerID="82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0" exitCode=0
Jul 06 19:50:39.025785 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.025774    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"
Jul 06 19:50:39.025785 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.025776    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv" event={"ID":"aeb9a8de-fbeb-4923-8824-2b7044af371a","Type":"ContainerDied","Data":"82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0"}
Jul 06 19:50:39.026016 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.025806    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv" event={"ID":"aeb9a8de-fbeb-4923-8824-2b7044af371a","Type":"ContainerDied","Data":"115bcb3e271d5c54c855282ddf48636f7a787fb69e570431b6b99a6c14f6f57f"}
Jul 06 19:50:39.033933 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.033913    2579 scope.go:117] "RemoveContainer" containerID="67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7"
Jul 06 19:50:39.034202 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:50:39.034160    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7\": container with ID starting with 67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7 not found: ID does not exist" containerID="67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7"
Jul 06 19:50:39.034202 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.034186    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7"} err="failed to get container status \"67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7\": rpc error: code = NotFound desc = could not find container \"67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7\": container with ID starting with 67cd8d945e6b397076fd3ee06e5a0697757b8acd7856b9a9124794cfc2cedfe7 not found: ID does not exist"
Jul 06 19:50:39.034202 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.034207    2579 scope.go:117] "RemoveContainer" containerID="82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0"
Jul 06 19:50:39.042022 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.041999    2579 scope.go:117] "RemoveContainer" containerID="82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0"
Jul 06 19:50:39.042339 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:50:39.042311    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0\": container with ID starting with 82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0 not found: ID does not exist" containerID="82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0"
Jul 06 19:50:39.042499 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.042350    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0"} err="failed to get container status \"82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0\": rpc error: code = NotFound desc = could not find container \"82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0\": container with ID starting with 82b8b9f3cf7bfc1a976359785a01d0a372cd126f02ec088cb43aba4f38aa5fb0 not found: ID does not exist"
Jul 06 19:50:39.044611 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.044572    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"]
Jul 06 19:50:39.046751 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.046732    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-5c92ad/rate-test-model-5c92ad-kserve-7b4b474675-d49hq"]
Jul 06 19:50:39.057082 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.057061    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"]
Jul 06 19:50:39.059211 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.059193    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-5f29b8/rate-test-model-5f29b8-kserve-f8bbf6747-fwwgv"]
Jul 06 19:50:39.765639 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.765598    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7551-140020564180688"]
Jul 06 19:50:39.766058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.765940    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7c92d8f-429b-4744-928b-a45c843d0d82" containerName="main"
Jul 06 19:50:39.766058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.765953    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7c92d8f-429b-4744-928b-a45c843d0d82" containerName="main"
Jul 06 19:50:39.766058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.765981    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aeb9a8de-fbeb-4923-8824-2b7044af371a" containerName="main"
Jul 06 19:50:39.766058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.765986    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="aeb9a8de-fbeb-4923-8824-2b7044af371a" containerName="main"
Jul 06 19:50:39.766058 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.766057    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="aeb9a8de-fbeb-4923-8824-2b7044af371a" containerName="main"
Jul 06 19:50:39.766223 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.766067    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7c92d8f-429b-4744-928b-a45c843d0d82" containerName="main"
Jul 06 19:50:39.770664 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.770639    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564180688"
Jul 06 19:50:39.773773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.773752    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-jlb4q\""
Jul 06 19:50:39.777407 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.777381    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564180688"]
Jul 06 19:50:39.863802 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.863771    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k5b4\" (UniqueName: \"kubernetes.io/projected/8a8c63ff-a781-42e6-a87a-f0e1ddcf020d-kube-api-access-2k5b4\") pod \"test-curl-7551-140020564180688\" (UID: \"8a8c63ff-a781-42e6-a87a-f0e1ddcf020d\") " pod="opendatahub/test-curl-7551-140020564180688"
Jul 06 19:50:39.964564 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.964526    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k5b4\" (UniqueName: \"kubernetes.io/projected/8a8c63ff-a781-42e6-a87a-f0e1ddcf020d-kube-api-access-2k5b4\") pod \"test-curl-7551-140020564180688\" (UID: \"8a8c63ff-a781-42e6-a87a-f0e1ddcf020d\") " pod="opendatahub/test-curl-7551-140020564180688"
Jul 06 19:50:39.973554 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:39.973519    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k5b4\" (UniqueName: \"kubernetes.io/projected/8a8c63ff-a781-42e6-a87a-f0e1ddcf020d-kube-api-access-2k5b4\") pod \"test-curl-7551-140020564180688\" (UID: \"8a8c63ff-a781-42e6-a87a-f0e1ddcf020d\") " pod="opendatahub/test-curl-7551-140020564180688"
Jul 06 19:50:40.081238 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:40.081201    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564180688"
Jul 06 19:50:40.220691 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:40.220667    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564180688"]
Jul 06 19:50:40.222712 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:50:40.222685    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a8c63ff_a781_42e6_a87a_f0e1ddcf020d.slice/crio-a574a2d10de964fcedbb77a9efa60817bdae1fc96acf117dfea97c8706688407 WatchSource:0}: Error finding container a574a2d10de964fcedbb77a9efa60817bdae1fc96acf117dfea97c8706688407: Status 404 returned error can't find the container with id a574a2d10de964fcedbb77a9efa60817bdae1fc96acf117dfea97c8706688407
Jul 06 19:50:40.223934 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:40.223916    2579 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 19:50:40.933749 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:40.933718    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aeb9a8de-fbeb-4923-8824-2b7044af371a" path="/var/lib/kubelet/pods/aeb9a8de-fbeb-4923-8824-2b7044af371a/volumes"
Jul 06 19:50:40.934220 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:40.934202    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7c92d8f-429b-4744-928b-a45c843d0d82" path="/var/lib/kubelet/pods/d7c92d8f-429b-4744-928b-a45c843d0d82/volumes"
Jul 06 19:50:41.039342 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:41.039313    2579 generic.go:356] "Generic (PLEG): container finished" podID="8a8c63ff-a781-42e6-a87a-f0e1ddcf020d" containerID="eb9dfbbcb4c4ad0afea8ccb34e9411d84b099a719a307d95a018315498410a48" exitCode=0
Jul 06 19:50:41.039483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:41.039362    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564180688" event={"ID":"8a8c63ff-a781-42e6-a87a-f0e1ddcf020d","Type":"ContainerDied","Data":"eb9dfbbcb4c4ad0afea8ccb34e9411d84b099a719a307d95a018315498410a48"}
Jul 06 19:50:41.039483 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:41.039390    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564180688" event={"ID":"8a8c63ff-a781-42e6-a87a-f0e1ddcf020d","Type":"ContainerStarted","Data":"a574a2d10de964fcedbb77a9efa60817bdae1fc96acf117dfea97c8706688407"}
Jul 06 19:50:42.175220 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.175196    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564180688"
Jul 06 19:50:42.181696 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.181650    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2k5b4\" (UniqueName: \"kubernetes.io/projected/8a8c63ff-a781-42e6-a87a-f0e1ddcf020d-kube-api-access-2k5b4\") pod \"8a8c63ff-a781-42e6-a87a-f0e1ddcf020d\" (UID: \"8a8c63ff-a781-42e6-a87a-f0e1ddcf020d\") "
Jul 06 19:50:42.183591 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.183570    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a8c63ff-a781-42e6-a87a-f0e1ddcf020d-kube-api-access-2k5b4" (OuterVolumeSpecName: "kube-api-access-2k5b4") pod "8a8c63ff-a781-42e6-a87a-f0e1ddcf020d" (UID: "8a8c63ff-a781-42e6-a87a-f0e1ddcf020d"). InnerVolumeSpecName "kube-api-access-2k5b4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:42.228291 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.228261    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7551-140020564180688_8a8c63ff-a781-42e6-a87a-f0e1ddcf020d/test-curl-7551-140020564180688/0.log"
Jul 06 19:50:42.257612 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.257582    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7551-140020564180688"]
Jul 06 19:50:42.263059 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.263031    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7551-140020564180688"]
Jul 06 19:50:42.282391 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.282362    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2k5b4\" (UniqueName: \"kubernetes.io/projected/8a8c63ff-a781-42e6-a87a-f0e1ddcf020d-kube-api-access-2k5b4\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:42.393388 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.393304    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:42.393652 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.393640    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a8c63ff-a781-42e6-a87a-f0e1ddcf020d" containerName="test-curl-7551-140020564180688"
Jul 06 19:50:42.393652 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.393666    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a8c63ff-a781-42e6-a87a-f0e1ddcf020d" containerName="test-curl-7551-140020564180688"
Jul 06 19:50:42.393750 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.393731    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a8c63ff-a781-42e6-a87a-f0e1ddcf020d" containerName="test-curl-7551-140020564180688"
Jul 06 19:50:42.396610 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.396594    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:42.404280 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.404256    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:42.484558 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.484515    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hz42h\" (UniqueName: \"kubernetes.io/projected/d46f81f3-f3fc-4829-a8ca-ab7d7f958976-kube-api-access-hz42h\") pod \"test-curl-7551-140020564262960\" (UID: \"d46f81f3-f3fc-4829-a8ca-ab7d7f958976\") " pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:42.585866 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.585826    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hz42h\" (UniqueName: \"kubernetes.io/projected/d46f81f3-f3fc-4829-a8ca-ab7d7f958976-kube-api-access-hz42h\") pod \"test-curl-7551-140020564262960\" (UID: \"d46f81f3-f3fc-4829-a8ca-ab7d7f958976\") " pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:42.595313 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.595279    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hz42h\" (UniqueName: \"kubernetes.io/projected/d46f81f3-f3fc-4829-a8ca-ab7d7f958976-kube-api-access-hz42h\") pod \"test-curl-7551-140020564262960\" (UID: \"d46f81f3-f3fc-4829-a8ca-ab7d7f958976\") " pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:42.706144 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.706058    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:42.838931 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.838901    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:42.840581 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:50:42.840554    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd46f81f3_f3fc_4829_a8ca_ab7d7f958976.slice/crio-a867c65e333d5ce6924193341d79855398519d1b8e4349cb0a601ef2b40eff10 WatchSource:0}: Error finding container a867c65e333d5ce6924193341d79855398519d1b8e4349cb0a601ef2b40eff10: Status 404 returned error can't find the container with id a867c65e333d5ce6924193341d79855398519d1b8e4349cb0a601ef2b40eff10
Jul 06 19:50:42.928121 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:42.928089    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a8c63ff-a781-42e6-a87a-f0e1ddcf020d" path="/var/lib/kubelet/pods/8a8c63ff-a781-42e6-a87a-f0e1ddcf020d/volumes"
Jul 06 19:50:43.050999 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:43.050976    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564180688"
Jul 06 19:50:43.051194 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:43.050975    2579 scope.go:117] "RemoveContainer" containerID="eb9dfbbcb4c4ad0afea8ccb34e9411d84b099a719a307d95a018315498410a48"
Jul 06 19:50:43.052417 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:43.052222    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564262960" event={"ID":"d46f81f3-f3fc-4829-a8ca-ab7d7f958976","Type":"ContainerStarted","Data":"a867c65e333d5ce6924193341d79855398519d1b8e4349cb0a601ef2b40eff10"}
Jul 06 19:50:44.060552 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:44.060521    2579 generic.go:356] "Generic (PLEG): container finished" podID="d46f81f3-f3fc-4829-a8ca-ab7d7f958976" containerID="f20b4c0a0d0e79e471f0340fa22a4adc42a5b86da666206df05f8abaac079ee2" exitCode=0
Jul 06 19:50:44.060946 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:44.060610    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564262960" event={"ID":"d46f81f3-f3fc-4829-a8ca-ab7d7f958976","Type":"ContainerDied","Data":"f20b4c0a0d0e79e471f0340fa22a4adc42a5b86da666206df05f8abaac079ee2"}
Jul 06 19:50:45.192980 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.192951    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:45.209400 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.209377    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hz42h\" (UniqueName: \"kubernetes.io/projected/d46f81f3-f3fc-4829-a8ca-ab7d7f958976-kube-api-access-hz42h\") pod \"d46f81f3-f3fc-4829-a8ca-ab7d7f958976\" (UID: \"d46f81f3-f3fc-4829-a8ca-ab7d7f958976\") "
Jul 06 19:50:45.211704 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.211670    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d46f81f3-f3fc-4829-a8ca-ab7d7f958976-kube-api-access-hz42h" (OuterVolumeSpecName: "kube-api-access-hz42h") pod "d46f81f3-f3fc-4829-a8ca-ab7d7f958976" (UID: "d46f81f3-f3fc-4829-a8ca-ab7d7f958976"). InnerVolumeSpecName "kube-api-access-hz42h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:45.218526 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.218502    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7551-140020564262960_d46f81f3-f3fc-4829-a8ca-ab7d7f958976/test-curl-7551-140020564262960/0.log"
Jul 06 19:50:45.245034 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.245005    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:45.251176 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.251154    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:45.310177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.310134    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hz42h\" (UniqueName: \"kubernetes.io/projected/d46f81f3-f3fc-4829-a8ca-ab7d7f958976-kube-api-access-hz42h\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:45.381365 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.381278    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:45.381615 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.381601    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d46f81f3-f3fc-4829-a8ca-ab7d7f958976" containerName="test-curl-7551-140020564262960"
Jul 06 19:50:45.381687 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.381615    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="d46f81f3-f3fc-4829-a8ca-ab7d7f958976" containerName="test-curl-7551-140020564262960"
Jul 06 19:50:45.381729 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.381700    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="d46f81f3-f3fc-4829-a8ca-ab7d7f958976" containerName="test-curl-7551-140020564262960"
Jul 06 19:50:45.384531 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.384515    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:45.394087 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.394061    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:45.411217 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.411187    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhzts\" (UniqueName: \"kubernetes.io/projected/8bf815fc-d6a0-4eeb-ad04-343098f5c896-kube-api-access-nhzts\") pod \"test-curl-7551-140020564262960\" (UID: \"8bf815fc-d6a0-4eeb-ad04-343098f5c896\") " pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:45.511904 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.511868    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhzts\" (UniqueName: \"kubernetes.io/projected/8bf815fc-d6a0-4eeb-ad04-343098f5c896-kube-api-access-nhzts\") pod \"test-curl-7551-140020564262960\" (UID: \"8bf815fc-d6a0-4eeb-ad04-343098f5c896\") " pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:45.522107 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.522082    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhzts\" (UniqueName: \"kubernetes.io/projected/8bf815fc-d6a0-4eeb-ad04-343098f5c896-kube-api-access-nhzts\") pod \"test-curl-7551-140020564262960\" (UID: \"8bf815fc-d6a0-4eeb-ad04-343098f5c896\") " pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:45.694193 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.694102    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:45.832135 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:45.832111    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:45.834714 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:50:45.834687    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bf815fc_d6a0_4eeb_ad04_343098f5c896.slice/crio-68b106579918797973fbede38389027929adaba6816bb22eb38db4f21d227d03 WatchSource:0}: Error finding container 68b106579918797973fbede38389027929adaba6816bb22eb38db4f21d227d03: Status 404 returned error can't find the container with id 68b106579918797973fbede38389027929adaba6816bb22eb38db4f21d227d03
Jul 06 19:50:46.076615 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:46.076581    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564262960" event={"ID":"8bf815fc-d6a0-4eeb-ad04-343098f5c896","Type":"ContainerStarted","Data":"68b106579918797973fbede38389027929adaba6816bb22eb38db4f21d227d03"}
Jul 06 19:50:46.077894 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:46.077868    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:46.078020 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:46.077905    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a867c65e333d5ce6924193341d79855398519d1b8e4349cb0a601ef2b40eff10"
Jul 06 19:50:46.090238 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:46.090209    2579 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7551-140020564262960" oldPodUID="d46f81f3-f3fc-4829-a8ca-ab7d7f958976" podUID="8bf815fc-d6a0-4eeb-ad04-343098f5c896"
Jul 06 19:50:46.927607 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:46.927573    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d46f81f3-f3fc-4829-a8ca-ab7d7f958976" path="/var/lib/kubelet/pods/d46f81f3-f3fc-4829-a8ca-ab7d7f958976/volumes"
Jul 06 19:50:47.084198 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:47.084163    2579 generic.go:356] "Generic (PLEG): container finished" podID="8bf815fc-d6a0-4eeb-ad04-343098f5c896" containerID="c27ac90be5267e1e571f15d86aa328b411bd9241f4b62f0adbfe3bf40ecab3ef" exitCode=0
Jul 06 19:50:47.084336 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:47.084248    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564262960" event={"ID":"8bf815fc-d6a0-4eeb-ad04-343098f5c896","Type":"ContainerDied","Data":"c27ac90be5267e1e571f15d86aa328b411bd9241f4b62f0adbfe3bf40ecab3ef"}
Jul 06 19:50:48.220004 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.219983    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:48.235717 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.235695    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhzts\" (UniqueName: \"kubernetes.io/projected/8bf815fc-d6a0-4eeb-ad04-343098f5c896-kube-api-access-nhzts\") pod \"8bf815fc-d6a0-4eeb-ad04-343098f5c896\" (UID: \"8bf815fc-d6a0-4eeb-ad04-343098f5c896\") "
Jul 06 19:50:48.237965 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.237938    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bf815fc-d6a0-4eeb-ad04-343098f5c896-kube-api-access-nhzts" (OuterVolumeSpecName: "kube-api-access-nhzts") pod "8bf815fc-d6a0-4eeb-ad04-343098f5c896" (UID: "8bf815fc-d6a0-4eeb-ad04-343098f5c896"). InnerVolumeSpecName "kube-api-access-nhzts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:48.242861 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.242841    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7551-140020564262960_8bf815fc-d6a0-4eeb-ad04-343098f5c896/test-curl-7551-140020564262960/0.log"
Jul 06 19:50:48.267609 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.267580    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:48.269605 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.269584    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7551-140020564262960"]
Jul 06 19:50:48.336472 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.336445    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhzts\" (UniqueName: \"kubernetes.io/projected/8bf815fc-d6a0-4eeb-ad04-343098f5c896-kube-api-access-nhzts\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:48.407275 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.407213    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7551-140020564140224"]
Jul 06 19:50:48.407540 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.407527    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bf815fc-d6a0-4eeb-ad04-343098f5c896" containerName="test-curl-7551-140020564262960"
Jul 06 19:50:48.407579 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.407541    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bf815fc-d6a0-4eeb-ad04-343098f5c896" containerName="test-curl-7551-140020564262960"
Jul 06 19:50:48.407620 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.407611    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bf815fc-d6a0-4eeb-ad04-343098f5c896" containerName="test-curl-7551-140020564262960"
Jul 06 19:50:48.410478 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.410462    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:48.418453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.418434    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564140224"]
Jul 06 19:50:48.437487 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.437465    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xpzd\" (UniqueName: \"kubernetes.io/projected/f877a4ef-e6d3-407c-91ff-9b7fcda0aad8-kube-api-access-8xpzd\") pod \"test-curl-7551-140020564140224\" (UID: \"f877a4ef-e6d3-407c-91ff-9b7fcda0aad8\") " pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:48.538851 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.538823    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xpzd\" (UniqueName: \"kubernetes.io/projected/f877a4ef-e6d3-407c-91ff-9b7fcda0aad8-kube-api-access-8xpzd\") pod \"test-curl-7551-140020564140224\" (UID: \"f877a4ef-e6d3-407c-91ff-9b7fcda0aad8\") " pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:48.549083 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.549057    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xpzd\" (UniqueName: \"kubernetes.io/projected/f877a4ef-e6d3-407c-91ff-9b7fcda0aad8-kube-api-access-8xpzd\") pod \"test-curl-7551-140020564140224\" (UID: \"f877a4ef-e6d3-407c-91ff-9b7fcda0aad8\") " pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:48.720318 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.720243    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:48.851746 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.851722    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564140224"]
Jul 06 19:50:48.853015 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:50:48.852994    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf877a4ef_e6d3_407c_91ff_9b7fcda0aad8.slice/crio-48261a2c7041456b2927f4b2163bf7b6da01c0184959a0f23cae7ea390fe15fb WatchSource:0}: Error finding container 48261a2c7041456b2927f4b2163bf7b6da01c0184959a0f23cae7ea390fe15fb: Status 404 returned error can't find the container with id 48261a2c7041456b2927f4b2163bf7b6da01c0184959a0f23cae7ea390fe15fb
Jul 06 19:50:48.927846 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:48.927817    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bf815fc-d6a0-4eeb-ad04-343098f5c896" path="/var/lib/kubelet/pods/8bf815fc-d6a0-4eeb-ad04-343098f5c896/volumes"
Jul 06 19:50:48.957538 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:50:48.957510    2579 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8bf815fc_d6a0_4eeb_ad04_343098f5c896.slice\": RecentStats: unable to find data in memory cache]"
Jul 06 19:50:49.095876 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:49.095845    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564262960"
Jul 06 19:50:49.096044 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:49.095853    2579 scope.go:117] "RemoveContainer" containerID="c27ac90be5267e1e571f15d86aa328b411bd9241f4b62f0adbfe3bf40ecab3ef"
Jul 06 19:50:49.097267 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:49.097068    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564140224" event={"ID":"f877a4ef-e6d3-407c-91ff-9b7fcda0aad8","Type":"ContainerStarted","Data":"48261a2c7041456b2927f4b2163bf7b6da01c0184959a0f23cae7ea390fe15fb"}
Jul 06 19:50:50.103775 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:50.103741    2579 generic.go:356] "Generic (PLEG): container finished" podID="f877a4ef-e6d3-407c-91ff-9b7fcda0aad8" containerID="50a9bf474b4fe499c444ec2382a65a614b1fc949e8d3b0cd7160c8bb98c510d3" exitCode=0
Jul 06 19:50:50.104133 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:50.103825    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564140224" event={"ID":"f877a4ef-e6d3-407c-91ff-9b7fcda0aad8","Type":"ContainerDied","Data":"50a9bf474b4fe499c444ec2382a65a614b1fc949e8d3b0cd7160c8bb98c510d3"}
Jul 06 19:50:51.239075 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.239052    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:51.258842 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.258808    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8xpzd\" (UniqueName: \"kubernetes.io/projected/f877a4ef-e6d3-407c-91ff-9b7fcda0aad8-kube-api-access-8xpzd\") pod \"f877a4ef-e6d3-407c-91ff-9b7fcda0aad8\" (UID: \"f877a4ef-e6d3-407c-91ff-9b7fcda0aad8\") "
Jul 06 19:50:51.261000 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.260976    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f877a4ef-e6d3-407c-91ff-9b7fcda0aad8-kube-api-access-8xpzd" (OuterVolumeSpecName: "kube-api-access-8xpzd") pod "f877a4ef-e6d3-407c-91ff-9b7fcda0aad8" (UID: "f877a4ef-e6d3-407c-91ff-9b7fcda0aad8"). InnerVolumeSpecName "kube-api-access-8xpzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:51.265398 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.265376    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7551-140020564140224_f877a4ef-e6d3-407c-91ff-9b7fcda0aad8/test-curl-7551-140020564140224/0.log"
Jul 06 19:50:51.297608 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.297585    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7551-140020564140224"]
Jul 06 19:50:51.301030 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.301011    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7551-140020564140224"]
Jul 06 19:50:51.360033 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.359979    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8xpzd\" (UniqueName: \"kubernetes.io/projected/f877a4ef-e6d3-407c-91ff-9b7fcda0aad8-kube-api-access-8xpzd\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:51.431836 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.431806    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7551-140020564140656"]
Jul 06 19:50:51.432193 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.432179    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f877a4ef-e6d3-407c-91ff-9b7fcda0aad8" containerName="test-curl-7551-140020564140224"
Jul 06 19:50:51.432293 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.432195    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="f877a4ef-e6d3-407c-91ff-9b7fcda0aad8" containerName="test-curl-7551-140020564140224"
Jul 06 19:50:51.432293 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.432263    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="f877a4ef-e6d3-407c-91ff-9b7fcda0aad8" containerName="test-curl-7551-140020564140224"
Jul 06 19:50:51.435151 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.435137    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140656"
Jul 06 19:50:51.445796 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.445773    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564140656"]
Jul 06 19:50:51.460813 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.460787    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpwvd\" (UniqueName: \"kubernetes.io/projected/1f971d16-4dba-465e-96ec-110c965db6bf-kube-api-access-qpwvd\") pod \"test-curl-7551-140020564140656\" (UID: \"1f971d16-4dba-465e-96ec-110c965db6bf\") " pod="opendatahub/test-curl-7551-140020564140656"
Jul 06 19:50:51.561903 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.561870    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpwvd\" (UniqueName: \"kubernetes.io/projected/1f971d16-4dba-465e-96ec-110c965db6bf-kube-api-access-qpwvd\") pod \"test-curl-7551-140020564140656\" (UID: \"1f971d16-4dba-465e-96ec-110c965db6bf\") " pod="opendatahub/test-curl-7551-140020564140656"
Jul 06 19:50:51.572967 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.572940    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpwvd\" (UniqueName: \"kubernetes.io/projected/1f971d16-4dba-465e-96ec-110c965db6bf-kube-api-access-qpwvd\") pod \"test-curl-7551-140020564140656\" (UID: \"1f971d16-4dba-465e-96ec-110c965db6bf\") " pod="opendatahub/test-curl-7551-140020564140656"
Jul 06 19:50:51.744463 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.744387    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140656"
Jul 06 19:50:51.906302 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:50:51.906272    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1f971d16_4dba_465e_96ec_110c965db6bf.slice/crio-ffc1b81a93a336b28671acc7dd0dd742011f838a115d4ed66f66d023a25106a8 WatchSource:0}: Error finding container ffc1b81a93a336b28671acc7dd0dd742011f838a115d4ed66f66d023a25106a8: Status 404 returned error can't find the container with id ffc1b81a93a336b28671acc7dd0dd742011f838a115d4ed66f66d023a25106a8
Jul 06 19:50:51.909040 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:51.909015    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7551-140020564140656"]
Jul 06 19:50:52.115326 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:52.115298    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:52.115509 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:52.115296    2579 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="48261a2c7041456b2927f4b2163bf7b6da01c0184959a0f23cae7ea390fe15fb"
Jul 06 19:50:52.119945 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:52.119913    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564140656" event={"ID":"1f971d16-4dba-465e-96ec-110c965db6bf","Type":"ContainerStarted","Data":"ffc1b81a93a336b28671acc7dd0dd742011f838a115d4ed66f66d023a25106a8"}
Jul 06 19:50:52.127809 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:50:52.127780    2579 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7551-140020564140224\" is forbidden: User \"system:node:ip-10-0-142-165.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-165.ec2.internal' and this object" podUID="f877a4ef-e6d3-407c-91ff-9b7fcda0aad8" pod="opendatahub/test-curl-7551-140020564140224"
Jul 06 19:50:52.927733 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:52.927702    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f877a4ef-e6d3-407c-91ff-9b7fcda0aad8" path="/var/lib/kubelet/pods/f877a4ef-e6d3-407c-91ff-9b7fcda0aad8/volumes"
Jul 06 19:50:53.127014 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:53.126979    2579 generic.go:356] "Generic (PLEG): container finished" podID="1f971d16-4dba-465e-96ec-110c965db6bf" containerID="36381bc769ebe69bb3e2335d5cfecf76a946f9d7e1394757bf651392b13ab4e8" exitCode=0
Jul 06 19:50:53.127216 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:53.127036    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7551-140020564140656" event={"ID":"1f971d16-4dba-465e-96ec-110c965db6bf","Type":"ContainerDied","Data":"36381bc769ebe69bb3e2335d5cfecf76a946f9d7e1394757bf651392b13ab4e8"}
Jul 06 19:50:54.266507 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.266484    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140656"
Jul 06 19:50:54.295390 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.295361    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7551-140020564140656_1f971d16-4dba-465e-96ec-110c965db6bf/test-curl-7551-140020564140656/0.log"
Jul 06 19:50:54.320036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.320014    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7551-140020564140656"]
Jul 06 19:50:54.327165 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.327146    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7551-140020564140656"]
Jul 06 19:50:54.384164 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.384080    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpwvd\" (UniqueName: \"kubernetes.io/projected/1f971d16-4dba-465e-96ec-110c965db6bf-kube-api-access-qpwvd\") pod \"1f971d16-4dba-465e-96ec-110c965db6bf\" (UID: \"1f971d16-4dba-465e-96ec-110c965db6bf\") "
Jul 06 19:50:54.386090 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.386060    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f971d16-4dba-465e-96ec-110c965db6bf-kube-api-access-qpwvd" (OuterVolumeSpecName: "kube-api-access-qpwvd") pod "1f971d16-4dba-465e-96ec-110c965db6bf" (UID: "1f971d16-4dba-465e-96ec-110c965db6bf"). InnerVolumeSpecName "kube-api-access-qpwvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:50:54.484882 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.484853    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpwvd\" (UniqueName: \"kubernetes.io/projected/1f971d16-4dba-465e-96ec-110c965db6bf-kube-api-access-qpwvd\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:50:54.928104 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:54.928069    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f971d16-4dba-465e-96ec-110c965db6bf" path="/var/lib/kubelet/pods/1f971d16-4dba-465e-96ec-110c965db6bf/volumes"
Jul 06 19:50:55.139805 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:55.139775    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7551-140020564140656"
Jul 06 19:50:55.139986 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:50:55.139823    2579 scope.go:117] "RemoveContainer" containerID="36381bc769ebe69bb3e2335d5cfecf76a946f9d7e1394757bf651392b13ab4e8"
Jul 06 19:51:18.970732 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:18.970695    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"]
Jul 06 19:51:18.971119 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:18.970908    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy" containerID="cri-o://dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5" gracePeriod=30
Jul 06 19:51:20.623223 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:20.623190    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"]
Jul 06 19:51:20.633497 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:20.633475    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-5f29b8-29722785-4772t"]
Jul 06 19:51:20.928927 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:20.928832    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96403e99-c032-46f7-80fa-df8bc13a28c1" path="/var/lib/kubelet/pods/96403e99-c032-46f7-80fa-df8bc13a28c1/volumes"
Jul 06 19:51:24.227973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.227951    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:51:24.295509 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.295476    2579 generic.go:356] "Generic (PLEG): container finished" podID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerID="dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5" exitCode=0
Jul 06 19:51:24.295705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.295517    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"
Jul 06 19:51:24.295705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.295542    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" event={"ID":"7e4c86e6-3a64-4227-89e4-da6961b83b50","Type":"ContainerDied","Data":"dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5"}
Jul 06 19:51:24.295705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.295601    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz" event={"ID":"7e4c86e6-3a64-4227-89e4-da6961b83b50","Type":"ContainerDied","Data":"feab26b3669ae4a469bc8bbdca239749b3f9ffe309a331169baca0b46c061e37"}
Jul 06 19:51:24.295705 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.295632    2579 scope.go:117] "RemoveContainer" containerID="dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5"
Jul 06 19:51:24.304222 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.304203    2579 scope.go:117] "RemoveContainer" containerID="b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e"
Jul 06 19:51:24.312129 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.312114    2579 scope.go:117] "RemoveContainer" containerID="dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5"
Jul 06 19:51:24.312350 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:51:24.312335    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5\": container with ID starting with dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5 not found: ID does not exist" containerID="dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5"
Jul 06 19:51:24.312413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.312357    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5"} err="failed to get container status \"dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5\": rpc error: code = NotFound desc = could not find container \"dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5\": container with ID starting with dc71176cde900c14d2592311d7852feacaf8a1266acffbb828c5faf4a8bb2bc5 not found: ID does not exist"
Jul 06 19:51:24.312413 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.312372    2579 scope.go:117] "RemoveContainer" containerID="b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e"
Jul 06 19:51:24.312590 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:51:24.312573    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e\": container with ID starting with b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e not found: ID does not exist" containerID="b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e"
Jul 06 19:51:24.312647 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.312600    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e"} err="failed to get container status \"b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e\": rpc error: code = NotFound desc = could not find container \"b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e\": container with ID starting with b1ea4de67d3908274ee527e219fd0a50784a4abdc3df4fbda76ccf8996028c5e not found: ID does not exist"
Jul 06 19:51:24.330490 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330466    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wgzj\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-kube-api-access-7wgzj\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330583 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330499    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-podinfo\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330583 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330534    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-certs\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330734 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330697    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-socket\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330774    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-token\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330806    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:24.330995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330823    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-envoy\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330866    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4c86e6-3a64-4227-89e4-da6961b83b50-istiod-ca-cert\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330891    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-credential-socket\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330929    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-data\") pod \"7e4c86e6-3a64-4227-89e4-da6961b83b50\" (UID: \"7e4c86e6-3a64-4227-89e4-da6961b83b50\") "
Jul 06 19:51:24.330995 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.330968    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:24.331453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.331160    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:24.331453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.331247    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.331453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.331265    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.331453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.331279    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.331453 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.331322    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e4c86e6-3a64-4227-89e4-da6961b83b50-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:51:24.331939 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.331915    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-data" (OuterVolumeSpecName: "istio-data") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:24.332827 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.332805    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:51:24.333079 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.333061    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-token" (OuterVolumeSpecName: "istio-token") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:51:24.333287 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.333270    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-kube-api-access-7wgzj" (OuterVolumeSpecName: "kube-api-access-7wgzj") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "kube-api-access-7wgzj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:51:24.333331 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.333316    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7e4c86e6-3a64-4227-89e4-da6961b83b50" (UID: "7e4c86e6-3a64-4227-89e4-da6961b83b50"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:24.432285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.432254    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wgzj\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-kube-api-access-7wgzj\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.432285 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.432284    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.432488 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.432298    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.432488 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.432309    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.432488 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.432318    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7e4c86e6-3a64-4227-89e4-da6961b83b50-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.432488 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.432325    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7e4c86e6-3a64-4227-89e4-da6961b83b50-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:24.625328 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.625291    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"]
Jul 06 19:51:24.627276 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.627254    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-5c92ad-openshift-default-f7b79649f-mkngz"]
Jul 06 19:51:24.927287 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:24.927211    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" path="/var/lib/kubelet/pods/7e4c86e6-3a64-4227-89e4-da6961b83b50/volumes"
Jul 06 19:51:27.260382 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:27.260349    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"]
Jul 06 19:51:27.260922 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:27.260571    2579 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy" containerID="cri-o://140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c" gracePeriod=30
Jul 06 19:51:27.928268 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:27.928234    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb5dc74cd-2hz4v_45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef/authorino/0.log"
Jul 06 19:51:32.501603 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.501579    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:51:32.603209 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603183    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-socket\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603229    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-envoy\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603248    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-data\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603269    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istiod-ca-cert\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603301    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-token\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603340 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603318    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5w4b8\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-kube-api-access-5w4b8\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603347    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-podinfo\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603371    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-certs\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603393    2579 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-credential-socket\") pod \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\" (UID: \"6e38678c-7b2c-494a-83ee-9abbc4f4f33d\") "
Jul 06 19:51:32.603556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603491    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:32.603803 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603684    2579 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.603803 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603712    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:32.603803 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603775    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:32.603989 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.603961    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 19:51:32.604118 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.604091    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-data" (OuterVolumeSpecName: "istio-data") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:32.605364 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.605340    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 19:51:32.605630 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.605603    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-token" (OuterVolumeSpecName: "istio-token") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:51:32.605822 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.605802    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 06 19:51:32.605928 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.605909    2579 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-kube-api-access-5w4b8" (OuterVolumeSpecName: "kube-api-access-5w4b8") pod "6e38678c-7b2c-494a-83ee-9abbc4f4f33d" (UID: "6e38678c-7b2c-494a-83ee-9abbc4f4f33d"). InnerVolumeSpecName "kube-api-access-5w4b8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 19:51:32.704407 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704376    2579 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-envoy\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.704407 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704401    2579 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-data\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.704577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704413    2579 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istiod-ca-cert\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.704577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704424    2579 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-token\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.704577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704438    2579 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5w4b8\" (UniqueName: \"kubernetes.io/projected/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-kube-api-access-5w4b8\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.704577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704451    2579 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-istio-podinfo\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.704577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704465    2579 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-workload-certs\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:32.704577 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:32.704477    2579 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6e38678c-7b2c-494a-83ee-9abbc4f4f33d-credential-socket\") on node \"ip-10-0-142-165.ec2.internal\" DevicePath \"\""
Jul 06 19:51:33.196026 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.195938    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722770-z2dzr_ee97f373-cd4a-4d37-8bcf-e3328d0350ea/cleanup/0.log"
Jul 06 19:51:33.307239 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.307214    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722785-ktdkt_806adc84-fd44-4c52-ac87-47e55e9b9fda/cleanup/0.log"
Jul 06 19:51:33.343721 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.343693    2579 generic.go:356] "Generic (PLEG): container finished" podID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerID="140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c" exitCode=0
Jul 06 19:51:33.343844 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.343742    2579 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"
Jul 06 19:51:33.343844 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.343784    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" event={"ID":"6e38678c-7b2c-494a-83ee-9abbc4f4f33d","Type":"ContainerDied","Data":"140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c"}
Jul 06 19:51:33.343844 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.343840    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4" event={"ID":"6e38678c-7b2c-494a-83ee-9abbc4f4f33d","Type":"ContainerDied","Data":"1d2468be73af467ef55754be591093c8dd18809820c106f5e33c35dc091b9238"}
Jul 06 19:51:33.343952 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.343865    2579 scope.go:117] "RemoveContainer" containerID="140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c"
Jul 06 19:51:33.352768 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.352752    2579 scope.go:117] "RemoveContainer" containerID="e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16"
Jul 06 19:51:33.360500 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.360480    2579 scope.go:117] "RemoveContainer" containerID="140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c"
Jul 06 19:51:33.360807 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:51:33.360787    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c\": container with ID starting with 140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c not found: ID does not exist" containerID="140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c"
Jul 06 19:51:33.360868 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.360813    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c"} err="failed to get container status \"140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c\": rpc error: code = NotFound desc = could not find container \"140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c\": container with ID starting with 140cecce0906768f237c9cc7026fa926baf7a7fd594260ba5d4419a20063cc4c not found: ID does not exist"
Jul 06 19:51:33.360868 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.360832    2579 scope.go:117] "RemoveContainer" containerID="e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16"
Jul 06 19:51:33.361084 ip-10-0-142-165 kubenswrapper[2579]: E0706 19:51:33.361047    2579 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16\": container with ID starting with e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16 not found: ID does not exist" containerID="e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16"
Jul 06 19:51:33.361134 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.361092    2579 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16"} err="failed to get container status \"e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16\": rpc error: code = NotFound desc = could not find container \"e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16\": container with ID starting with e62b04661a1587e6ea79a550e1e9209daa69dcb47a83dc5e4f30a79e4f0f3e16 not found: ID does not exist"
Jul 06 19:51:33.365169 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.365149    2579 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"]
Jul 06 19:51:33.374187 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.374165    2579 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-5f29b8-openshift-default-f56db9bbc-ktgn4"]
Jul 06 19:51:33.432365 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.432344    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7558657f96-wwrjg_457f7f2d-3135-4e8c-9fcd-d8d0641a8314/manager/0.log"
Jul 06 19:51:33.690172 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:33.690126    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b88jv_bbf6f300-9954-49ff-8236-e44e0c7dfde2/manager/4.log"
Jul 06 19:51:34.075373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:34.075342    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_66e3b8e2-2424-4b25-91e1-d04f448437c3/copy-manifests/0.log"
Jul 06 19:51:34.082914 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:34.082885    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_66e3b8e2-2424-4b25-91e1-d04f448437c3/manager/3.log"
Jul 06 19:51:34.928123 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:34.928087    2579 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" path="/var/lib/kubelet/pods/6e38678c-7b2c-494a-83ee-9abbc4f4f33d/volumes"
Jul 06 19:51:34.992354 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:34.992332    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff_88dd27cd-dc98-4750-9d67-917967b92b26/util/0.log"
Jul 06 19:51:35.007298 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.007278    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff_88dd27cd-dc98-4750-9d67-917967b92b26/pull/0.log"
Jul 06 19:51:35.028336 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.028313    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff_88dd27cd-dc98-4750-9d67-917967b92b26/extract/0.log"
Jul 06 19:51:35.148968 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.148938    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb_7d8b70a3-4238-4c95-85ba-173343a207fe/pull/0.log"
Jul 06 19:51:35.157962 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.157942    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb_7d8b70a3-4238-4c95-85ba-173343a207fe/extract/0.log"
Jul 06 19:51:35.165026 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.165006    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb_7d8b70a3-4238-4c95-85ba-173343a207fe/util/0.log"
Jul 06 19:51:35.286888 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.286868    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj_273c7612-13e3-43df-9c8b-d46a8a5a7d52/util/0.log"
Jul 06 19:51:35.294901 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.294877    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj_273c7612-13e3-43df-9c8b-d46a8a5a7d52/pull/0.log"
Jul 06 19:51:35.302538 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.302520    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj_273c7612-13e3-43df-9c8b-d46a8a5a7d52/extract/0.log"
Jul 06 19:51:35.418127 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.418107    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9_c4b70b22-d35e-4d82-aa8e-faa5bebad0c1/extract/0.log"
Jul 06 19:51:35.427167 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.427143    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9_c4b70b22-d35e-4d82-aa8e-faa5bebad0c1/util/0.log"
Jul 06 19:51:35.434936 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.434918    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9_c4b70b22-d35e-4d82-aa8e-faa5bebad0c1/pull/0.log"
Jul 06 19:51:35.559330 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:35.559269    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb5dc74cd-2hz4v_45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef/authorino/0.log"
Jul 06 19:51:36.966241 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:36.966199    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9545dd54-rhcsh_8e455cae-324d-4ec9-9aff-12704be55ebe/kube-auth-proxy/0.log"
Jul 06 19:51:37.323064 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:37.323037    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-94rq2_ad2eb8e7-622d-436e-97a8-171df0afc050/payload-pre-processing/0.log"
Jul 06 19:51:37.450700 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:37.450668    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ppprd_5d97bef7-0442-4148-975d-33e45d3b6572/payload-processing/0.log"
Jul 06 19:51:37.930375 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:37.930342    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bk8bq_0e084384-b9cb-4e97-acb6-cff203045fc1/main/0.log"
Jul 06 19:51:43.201484 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201448    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqdz2/must-gather-npg5d"]
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201787    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f971d16-4dba-465e-96ec-110c965db6bf" containerName="test-curl-7551-140020564140656"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201800    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f971d16-4dba-465e-96ec-110c965db6bf" containerName="test-curl-7551-140020564140656"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201819    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201824    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201837    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201843    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201850    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201857    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201866    2579 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy"
Jul 06 19:51:43.201906 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201874    2579 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy"
Jul 06 19:51:43.202534 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201921    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy"
Jul 06 19:51:43.202534 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201931    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy"
Jul 06 19:51:43.202534 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201940    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f971d16-4dba-465e-96ec-110c965db6bf" containerName="test-curl-7551-140020564140656"
Jul 06 19:51:43.202534 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.201987    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e38678c-7b2c-494a-83ee-9abbc4f4f33d" containerName="istio-proxy"
Jul 06 19:51:43.202534 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.202179    2579 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e4c86e6-3a64-4227-89e4-da6961b83b50" containerName="istio-proxy"
Jul 06 19:51:43.205324 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.205309    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.208124 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.208107    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqdz2\"/\"kube-root-ca.crt\""
Jul 06 19:51:43.208211 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.208190    2579 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nqdz2\"/\"openshift-service-ca.crt\""
Jul 06 19:51:43.221036 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.221011    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqdz2/must-gather-npg5d"]
Jul 06 19:51:43.392365 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.392333    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/36e2b8b2-385f-4a12-ad72-782735ed7478-must-gather-output\") pod \"must-gather-npg5d\" (UID: \"36e2b8b2-385f-4a12-ad72-782735ed7478\") " pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.392498 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.392416    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhr5s\" (UniqueName: \"kubernetes.io/projected/36e2b8b2-385f-4a12-ad72-782735ed7478-kube-api-access-mhr5s\") pod \"must-gather-npg5d\" (UID: \"36e2b8b2-385f-4a12-ad72-782735ed7478\") " pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.492938 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.492859    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhr5s\" (UniqueName: \"kubernetes.io/projected/36e2b8b2-385f-4a12-ad72-782735ed7478-kube-api-access-mhr5s\") pod \"must-gather-npg5d\" (UID: \"36e2b8b2-385f-4a12-ad72-782735ed7478\") " pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.492938 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.492907    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/36e2b8b2-385f-4a12-ad72-782735ed7478-must-gather-output\") pod \"must-gather-npg5d\" (UID: \"36e2b8b2-385f-4a12-ad72-782735ed7478\") " pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.493193 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.493176    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/36e2b8b2-385f-4a12-ad72-782735ed7478-must-gather-output\") pod \"must-gather-npg5d\" (UID: \"36e2b8b2-385f-4a12-ad72-782735ed7478\") " pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.508431 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.508403    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhr5s\" (UniqueName: \"kubernetes.io/projected/36e2b8b2-385f-4a12-ad72-782735ed7478-kube-api-access-mhr5s\") pod \"must-gather-npg5d\" (UID: \"36e2b8b2-385f-4a12-ad72-782735ed7478\") " pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.514294 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.514274    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqdz2/must-gather-npg5d"
Jul 06 19:51:43.653280 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:43.653256    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqdz2/must-gather-npg5d"]
Jul 06 19:51:43.655348 ip-10-0-142-165 kubenswrapper[2579]: W0706 19:51:43.655324    2579 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36e2b8b2_385f_4a12_ad72_782735ed7478.slice/crio-ebd630d717f921d1f521482b22674ed6f919c133cbb2dfbdbfe4590720a24150 WatchSource:0}: Error finding container ebd630d717f921d1f521482b22674ed6f919c133cbb2dfbdbfe4590720a24150: Status 404 returned error can't find the container with id ebd630d717f921d1f521482b22674ed6f919c133cbb2dfbdbfe4590720a24150
Jul 06 19:51:44.404095 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:44.404070    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqdz2/must-gather-npg5d" event={"ID":"36e2b8b2-385f-4a12-ad72-782735ed7478","Type":"ContainerStarted","Data":"ebd630d717f921d1f521482b22674ed6f919c133cbb2dfbdbfe4590720a24150"}
Jul 06 19:51:45.413361 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:45.413322    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqdz2/must-gather-npg5d" event={"ID":"36e2b8b2-385f-4a12-ad72-782735ed7478","Type":"ContainerStarted","Data":"4c142f49421600a70582c78941ed56512e80ab1aa389f905c3d79ad6caed7a35"}
Jul 06 19:51:45.413851 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:45.413375    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqdz2/must-gather-npg5d" event={"ID":"36e2b8b2-385f-4a12-ad72-782735ed7478","Type":"ContainerStarted","Data":"edf28f83ab1f41bf6d0a0d106b011ea49aa5fefb4418150f961a9d30ed845cdf"}
Jul 06 19:51:45.434998 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:45.434936    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqdz2/must-gather-npg5d" podStartSLOduration=1.722152397 podStartE2EDuration="2.434913814s" totalImagesPullingTime="712.761417ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 19:51:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:45.433527336 +0000 UTC m=+3165.044852872" watchObservedRunningTime="2026-07-06 19:51:45.434913814 +0000 UTC m=+3165.046239352"
Jul 06 19:51:46.370356 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:46.370318    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zmcn6_03aed337-be99-4849-94f5-0cdeeab4857f/global-pull-secret-syncer/0.log"
Jul 06 19:51:46.575978 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:46.575951    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-jgr24_2d3ea8d3-5393-4900-8d57-3cc39e7fca96/konnectivity-agent/0.log"
Jul 06 19:51:46.676643 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:46.676571    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-165.ec2.internal_7a9b387f1ea494ba8e3440d0784de7c5/haproxy/0.log"
Jul 06 19:51:51.002108 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.002071    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff_88dd27cd-dc98-4750-9d67-917967b92b26/extract/0.log"
Jul 06 19:51:51.049177 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.049143    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff_88dd27cd-dc98-4750-9d67-917967b92b26/util/0.log"
Jul 06 19:51:51.127306 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.127269    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mb9ff_88dd27cd-dc98-4750-9d67-917967b92b26/pull/0.log"
Jul 06 19:51:51.182649 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.182572    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb_7d8b70a3-4238-4c95-85ba-173343a207fe/extract/0.log"
Jul 06 19:51:51.221559 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.221533    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb_7d8b70a3-4238-4c95-85ba-173343a207fe/util/0.log"
Jul 06 19:51:51.260408 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.260330    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0xjkbb_7d8b70a3-4238-4c95-85ba-173343a207fe/pull/0.log"
Jul 06 19:51:51.300969 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.300943    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj_273c7612-13e3-43df-9c8b-d46a8a5a7d52/extract/0.log"
Jul 06 19:51:51.335024 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.334998    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj_273c7612-13e3-43df-9c8b-d46a8a5a7d52/util/0.log"
Jul 06 19:51:51.366751 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.366718    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737phcj_273c7612-13e3-43df-9c8b-d46a8a5a7d52/pull/0.log"
Jul 06 19:51:51.422148 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.422113    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9_c4b70b22-d35e-4d82-aa8e-faa5bebad0c1/extract/0.log"
Jul 06 19:51:51.454733 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.454697    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9_c4b70b22-d35e-4d82-aa8e-faa5bebad0c1/util/0.log"
Jul 06 19:51:51.498297 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.498269    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ssln9_c4b70b22-d35e-4d82-aa8e-faa5bebad0c1/pull/0.log"
Jul 06 19:51:51.983315 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:51.983139    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-5cb5dc74cd-2hz4v_45d2a523-0ce6-4f3b-bdfe-9ff157f1f5ef/authorino/0.log"
Jul 06 19:51:54.732487 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:54.732443    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cgm7z_0675175b-5f36-4bab-8c2d-f50d50443d39/node-exporter/0.log"
Jul 06 19:51:54.765608 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:54.765574    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cgm7z_0675175b-5f36-4bab-8c2d-f50d50443d39/kube-rbac-proxy/0.log"
Jul 06 19:51:54.811589 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:54.811556    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cgm7z_0675175b-5f36-4bab-8c2d-f50d50443d39/init-textfile/0.log"
Jul 06 19:51:56.215390 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.215352    2579 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"]
Jul 06 19:51:56.225737 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.225711    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.228973 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.228937    2579 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nqdz2\"/\"default-dockercfg-6xq4g\""
Jul 06 19:51:56.249871 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.249831    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"]
Jul 06 19:51:56.315556 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.315526    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-sys\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.315773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.315575    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-podres\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.315773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.315648    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-proc\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.315773 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.315718    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-lib-modules\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.315962 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.315801    2579 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfx25\" (UniqueName: \"kubernetes.io/projected/e419c90c-8035-43b2-b23d-5ca3ce884b30-kube-api-access-jfx25\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.416842 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.416739    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfx25\" (UniqueName: \"kubernetes.io/projected/e419c90c-8035-43b2-b23d-5ca3ce884b30-kube-api-access-jfx25\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.416842 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.416783    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-sys\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.416842 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.416825    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-podres\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.417137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.416858    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-proc\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.417137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.416888    2579 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-lib-modules\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.417137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.416968    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-proc\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.417137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.417009    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-podres\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.417137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.417042    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-lib-modules\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.417137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.417058    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e419c90c-8035-43b2-b23d-5ca3ce884b30-sys\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.428244 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.428208    2579 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfx25\" (UniqueName: \"kubernetes.io/projected/e419c90c-8035-43b2-b23d-5ca3ce884b30-kube-api-access-jfx25\") pod \"perf-node-gather-daemonset-qhwwn\" (UID: \"e419c90c-8035-43b2-b23d-5ca3ce884b30\") " pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.540364 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.540327    2579 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:56.725683 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:56.725587    2579 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"]
Jul 06 19:51:57.025489 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:57.025412    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6bffdf9778-xhr6r_a54d672b-7076-4068-92ab-5e1a4406abc9/networking-console-plugin/0.log"
Jul 06 19:51:57.506311 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:57.506266    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn" event={"ID":"e419c90c-8035-43b2-b23d-5ca3ce884b30","Type":"ContainerStarted","Data":"a7324b2e448538456b5fa53fd69ca5e911932e5e29037bb73026bbbeed685a60"}
Jul 06 19:51:57.506721 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:57.506331    2579 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn" event={"ID":"e419c90c-8035-43b2-b23d-5ca3ce884b30","Type":"ContainerStarted","Data":"113eac25a0d85bb4e5485b091535df4af83f54635ba5971d554d26caf6e5b371"}
Jul 06 19:51:57.506721 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:57.506407    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:51:57.544641 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:57.544575    2579 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn" podStartSLOduration=1.544553952 podStartE2EDuration="1.544553952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 19:51:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 19:51:57.542198788 +0000 UTC m=+3177.153524318" watchObservedRunningTime="2026-07-06 19:51:57.544553952 +0000 UTC m=+3177.155879491"
Jul 06 19:51:58.093109 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:51:58.093076    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-bc77fb5b9-vk7zn_25b105dd-b380-47e1-9526-cc45814257a8/console/0.log"
Jul 06 19:52:00.083719 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:00.083691    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2ll9p_eda345d4-11ac-4c1e-9e3b-1607b869c7c2/dns/0.log"
Jul 06 19:52:00.111790 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:00.111763    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2ll9p_eda345d4-11ac-4c1e-9e3b-1607b869c7c2/kube-rbac-proxy/0.log"
Jul 06 19:52:00.308921 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:00.308888    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-twkgb_7f8025aa-247e-4f49-b65e-0830d654dc2b/dns-node-resolver/0.log"
Jul 06 19:52:00.915875 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:00.915791    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56f9b5f9f8-rqzvd_92912c24-3423-4e41-bc36-49089b0ca461/registry/0.log"
Jul 06 19:52:00.946063 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:00.946033    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cxvwk_7c7a182e-597b-43d5-9c20-9e10ee7422c6/node-ca/0.log"
Jul 06 19:52:01.276319 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:01.276211    2579 scope.go:117] "RemoveContainer" containerID="f32c285e9ca073698a2f2b8e1c65ffb1c821130a1165d63a9e313e7529c070bf"
Jul 06 19:52:02.210137 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:02.210057    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9545dd54-rhcsh_8e455cae-324d-4ec9-9aff-12704be55ebe/kube-auth-proxy/0.log"
Jul 06 19:52:02.356313 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:02.356280    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-94rq2_ad2eb8e7-622d-436e-97a8-171df0afc050/payload-pre-processing/0.log"
Jul 06 19:52:02.440373 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:02.440338    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ppprd_5d97bef7-0442-4148-975d-33e45d3b6572/payload-processing/0.log"
Jul 06 19:52:03.380390 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:03.380356    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2nz9k_d822dfd6-b5ec-4a8e-9fee-f5797dddac60/serve-healthcheck-canary/0.log"
Jul 06 19:52:03.528088 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:03.528055    2579 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nqdz2/perf-node-gather-daemonset-qhwwn"
Jul 06 19:52:04.063854 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:04.063824    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8fsz_7bbdad23-8c7e-49c3-ab8a-de9caced9c31/kube-rbac-proxy/0.log"
Jul 06 19:52:04.091091 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:04.091065    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8fsz_7bbdad23-8c7e-49c3-ab8a-de9caced9c31/exporter/0.log"
Jul 06 19:52:04.115302 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:04.115274    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-q8fsz_7bbdad23-8c7e-49c3-ab8a-de9caced9c31/extractor/0.log"
Jul 06 19:52:06.298772 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:06.298737    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9545dd54-rhcsh_8e455cae-324d-4ec9-9aff-12704be55ebe/kube-auth-proxy/0.log"
Jul 06 19:52:06.424089 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:06.424060    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-94rq2_ad2eb8e7-622d-436e-97a8-171df0afc050/payload-pre-processing/0.log"
Jul 06 19:52:06.493155 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:06.493119    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ppprd_5d97bef7-0442-4148-975d-33e45d3b6572/payload-processing/0.log"
Jul 06 19:52:07.127421 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.127392    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722770-z2dzr_ee97f373-cd4a-4d37-8bcf-e3328d0350ea/cleanup/0.log"
Jul 06 19:52:07.158118 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.158087    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722785-ktdkt_806adc84-fd44-4c52-ac87-47e55e9b9fda/cleanup/0.log"
Jul 06 19:52:07.344281 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.344247    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7558657f96-wwrjg_457f7f2d-3135-4e8c-9fcd-d8d0641a8314/manager/0.log"
Jul 06 19:52:07.405115 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.405031    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b88jv_bbf6f300-9954-49ff-8236-e44e0c7dfde2/manager/3.log"
Jul 06 19:52:07.465880 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.465849    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-b88jv_bbf6f300-9954-49ff-8236-e44e0c7dfde2/manager/4.log"
Jul 06 19:52:07.666582 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.666503    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_66e3b8e2-2424-4b25-91e1-d04f448437c3/manager/2.log"
Jul 06 19:52:07.667843 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.667816    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_66e3b8e2-2424-4b25-91e1-d04f448437c3/manager/3.log"
Jul 06 19:52:07.685488 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.685434    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d5dcdfbc-c7vs4_66e3b8e2-2424-4b25-91e1-d04f448437c3/copy-manifests/0.log"
Jul 06 19:52:07.862977 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.862942    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7c9545dd54-rhcsh_8e455cae-324d-4ec9-9aff-12704be55ebe/kube-auth-proxy/0.log"
Jul 06 19:52:07.985829 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:07.985753    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-94rq2_ad2eb8e7-622d-436e-97a8-171df0afc050/payload-pre-processing/0.log"
Jul 06 19:52:08.059612 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:08.059582    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-ppprd_5d97bef7-0442-4148-975d-33e45d3b6572/payload-processing/0.log"
Jul 06 19:52:09.106932 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:09.106902    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84979c6fd9-757nm_1f2ac7c7-46df-4479-a0fc-bd3b25ad8787/manager/2.log"
Jul 06 19:52:09.108239 ip-10-0-142-165 kubenswrapper[2579]: I0706 19:52:09.108214    2579 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-84979c6fd9-757nm_1f2ac7c7-46df-4479-a0fc-bd3b25ad8787/manager/1.log"