Jul 23 17:42:57.066667 ip-10-0-139-151 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 23 17:42:57.066678 ip-10-0-139-151 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 23 17:42:57.066685 ip-10-0-139-151 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 23 17:42:57.066935 ip-10-0-139-151 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 23 17:43:07.114677 ip-10-0-139-151 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 23 17:43:07.114698 ip-10-0-139-151 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e5c05d2aa0c241f18773491fc580fde6 --
Jul 23 17:45:05.024091 ip-10-0-139-151 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 17:45:05.456728 ip-10-0-139-151 kubenswrapper[2582]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 17:45:05.456728 ip-10-0-139-151 kubenswrapper[2582]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 17:45:05.456728 ip-10-0-139-151 kubenswrapper[2582]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 17:45:05.456728 ip-10-0-139-151 kubenswrapper[2582]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 17:45:05.456728 ip-10-0-139-151 kubenswrapper[2582]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 17:45:05.459253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.459156    2582 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 17:45:05.465200 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465179    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 17:45:05.465200 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465199    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 17:45:05.465200 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465203    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465208    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465211    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465215    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465219    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465223    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465227    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465230    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465233    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465236    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465239    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465242    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465246    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465248    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465251    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465254    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465263    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465266    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465269    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 17:45:05.465303 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465272    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465274    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465277    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465280    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465282    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465285    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465287    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465290    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465293    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465295    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465298    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465300    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465303    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465306    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465309    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465312    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465314    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465319    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465323    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465326    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 17:45:05.465757 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465329    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465332    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465334    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465338    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465340    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465343    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465346    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465348    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465351    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465353    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465356    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465359    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465361    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465364    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465367    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465370    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465373    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465376    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465379    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465382    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 17:45:05.466235 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465385    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465387    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465390    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465392    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465395    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465397    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465400    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465402    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465405    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465407    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465410    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465412    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465417    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465420    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465422    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465425    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465428    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465430    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465433    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465436    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 17:45:05.466750 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465438    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465441    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465443    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465446    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465448    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465451    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465455    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465459    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465463    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465466    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465469    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465472    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465475    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465478    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465481    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465484    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465487    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.465489    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466223    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 17:45:05.467255 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466229    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466232    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466235    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466238    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466241    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466244    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466247    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466250    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466253    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466256    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466259    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466262    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466264    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466267    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466271    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466275    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466278    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466281    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466283    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466287    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 17:45:05.467974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466290    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466292    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466295    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466298    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466301    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466303    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466306    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466309    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466311    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466314    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466317    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466319    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466322    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466324    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466327    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466329    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466332    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466334    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466337    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 17:45:05.468512 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466339    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466342    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466346    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466348    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466352    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466354    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466357    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466360    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466362    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466365    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466367    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466370    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466373    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466376    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466378    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466380    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466383    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466386    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466398    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466401    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 17:45:05.468992 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466403    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466406    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466409    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466411    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466414    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466417    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466420    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466422    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466425    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466428    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466431    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466434    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466436    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466438    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466441    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466444    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466446    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466449    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466452    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466454    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 17:45:05.469473 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466460    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466462    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466465    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466467    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466470    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466474    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466478    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466481    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466483    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466486    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466488    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466491    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466494    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466497    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466501    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466504    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466507    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466510    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466513    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 17:45:05.469971 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466919    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466924    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466927    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466930    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466934    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466937    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466941    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466945    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466948    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.466951    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467001    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467074    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467082    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467090    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467094    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467099    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467103    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467109    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467113    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 17:45:05.470429 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467118    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467123    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467128    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467132    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467146    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467235    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467240    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467244    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467248    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467253    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467257    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467262    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467266    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467271    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467276    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467280    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467290    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467294    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467299    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 17:45:05.470930 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467305    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467311    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467316    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467321    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467326    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467332    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467338    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467343    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467347    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467352    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467361    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467365    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467370    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467374    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467378    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467382    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467386    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467391    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467395    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467399    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 17:45:05.471388 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467403    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467408    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467416    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467420    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467424    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467428    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467433    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467437    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467441    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467445    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467449    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467453    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467457    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467461    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467465    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467473    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467478    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467482    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467487    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467494    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 17:45:05.471910 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467498    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467505    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467509    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467513    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467517    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467522    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467526    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467535    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467539    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467543    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467547    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467551    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467555    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467560    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467564    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467569    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467573    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467577    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467581    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467590    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 17:45:05.472385 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.467594    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467755    2582 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467766    2582 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467777    2582 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467784    2582 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467791    2582 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467802    2582 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467809    2582 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467817    2582 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467822    2582 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467827    2582 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467834    2582 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467840    2582 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467847    2582 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467852    2582 flags.go:64] FLAG: --cgroup-root=""
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467862    2582 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467867    2582 flags.go:64] FLAG: --client-ca-file=""
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467872    2582 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467877    2582 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467884    2582 flags.go:64] FLAG: --cluster-domain=""
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467889    2582 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467894    2582 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467899    2582 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467909    2582 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 17:45:05.472881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467915    2582 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467921    2582 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467926    2582 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467931    2582 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467937    2582 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467942    2582 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467947    2582 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467952    2582 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467962    2582 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467968    2582 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467973    2582 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467978    2582 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467983    2582 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467989    2582 flags.go:64] FLAG: --enable-server="true"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.467994    2582 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468451    2582 flags.go:64] FLAG: --event-burst="100"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468459    2582 flags.go:64] FLAG: --event-qps="50"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468467    2582 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468578    2582 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468584    2582 flags.go:64] FLAG: --eviction-hard=""
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468605    2582 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468608    2582 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468612    2582 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468616    2582 flags.go:64] FLAG: --eviction-soft=""
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468619    2582 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 17:45:05.473487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468622    2582 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468625    2582 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468628    2582 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468632    2582 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468635    2582 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468638    2582 flags.go:64] FLAG: --feature-gates=""
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468642    2582 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468645    2582 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468648    2582 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468652    2582 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468656    2582 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468659    2582 flags.go:64] FLAG: --help="false"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468662    2582 flags.go:64] FLAG: --hostname-override="ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468665    2582 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468668    2582 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468673    2582 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468677    2582 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468681    2582 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468684    2582 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468687    2582 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468690    2582 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468693    2582 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468697    2582 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468700    2582 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 17:45:05.474115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468718    2582 flags.go:64] FLAG: --kube-reserved=""
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468723    2582 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468727    2582 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468730    2582 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468734    2582 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468737    2582 flags.go:64] FLAG: --lock-file=""
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468740    2582 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468743    2582 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468746    2582 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468752    2582 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468755    2582 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468758    2582 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468761    2582 flags.go:64] FLAG: --logging-format="text"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468765    2582 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468768    2582 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468771    2582 flags.go:64] FLAG: --manifest-url=""
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468774    2582 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468779    2582 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468782    2582 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468787    2582 flags.go:64] FLAG: --max-pods="110"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468790    2582 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468794    2582 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468797    2582 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468800    2582 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468803    2582 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 17:45:05.474866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468806    2582 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468810    2582 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468819    2582 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468823    2582 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468828    2582 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468832    2582 flags.go:64] FLAG: --pod-cidr=""
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468835    2582 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468842    2582 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468845    2582 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468848    2582 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468851    2582 flags.go:64] FLAG: --port="10250"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468854    2582 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468857    2582 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c6389584851afc39"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468861    2582 flags.go:64] FLAG: --qos-reserved=""
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468864    2582 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468867    2582 flags.go:64] FLAG: --register-node="true"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468870    2582 flags.go:64] FLAG: --register-with-taints=""
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468874    2582 flags.go:64] FLAG: --registry-burst="10"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468876    2582 flags.go:64] FLAG: --registry-qps="5"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468879    2582 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468882    2582 flags.go:64] FLAG: --reserved-memory=""
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468886    2582 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468889    2582 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468892    2582 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468895    2582 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 17:45:05.475477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468898    2582 flags.go:64] FLAG: --runonce="false"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468901    2582 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468904    2582 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468907    2582 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468911    2582 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468914    2582 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468918    2582 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468921    2582 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468924    2582 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468927    2582 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468930    2582 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468933    2582 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468938    2582 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468941    2582 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468944    2582 flags.go:64] FLAG: --system-cgroups=""
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468946    2582 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468955    2582 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468958    2582 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468961    2582 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468966    2582 flags.go:64] FLAG: --tls-min-version=""
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468969    2582 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468972    2582 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468975    2582 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468978    2582 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468982    2582 flags.go:64] FLAG: --v="2"
Jul 23 17:45:05.476513 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468986    2582 flags.go:64] FLAG: --version="false"
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468990    2582 flags.go:64] FLAG: --vmodule=""
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468994    2582 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.468998    2582 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469123    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469127    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469131    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469134    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469136    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469139    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469142    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469145    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469149    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469153    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469155    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469158    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469160    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469163    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469166    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469170    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 17:45:05.477584 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469173    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469176    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469179    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469182    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469184    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469187    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469190    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469193    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469195    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469198    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469200    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469203    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469205    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469208    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469211    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469213    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469216    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469218    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469221    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469223    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 17:45:05.478524 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469226    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469229    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469231    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469234    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469237    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469240    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469243    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469246    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469249    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469251    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469256    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469262    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469267    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469270    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469273    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469276    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469279    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469282    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469284    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 17:45:05.479545 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469287    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469290    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469292    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469295    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469298    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469300    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469303    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469305    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469308    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469310    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469313    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469316    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469318    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469321    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469323    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469326    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469329    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469331    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469339    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 17:45:05.480152 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469343    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469345    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469349    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469351    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469354    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469358    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469360    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469363    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469366    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469370    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469373    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469377    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469379    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469382    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469386    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469389    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469392    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469395    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469398    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469401    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 17:45:05.480962 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469404    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469407    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469409    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469412    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.469415    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.470037    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.477909    2582 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.477932    2582 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.477963    2582 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.477972    2582 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478064    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478072    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478078    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478082    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 17:45:05.481747 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478087    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478091    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478096    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478100    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478104    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478109    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478116    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478122    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478127    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478132    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478136    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478141    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478146    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478150    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478154    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478159    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478163    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478168    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478172    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 17:45:05.482669 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478176    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478180    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478185    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478189    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478193    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478197    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478204    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478208    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478213    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478217    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478222    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478226    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478231    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478235    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478239    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478244    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478248    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478252    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478256    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478262    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 17:45:05.483300 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478269    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478273    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478278    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478282    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478287    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478291    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478295    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478299    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478303    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478308    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478312    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478317    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478321    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478325    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478330    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478334    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478338    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478342    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478347    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478352    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 17:45:05.483869 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478356    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478361    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478365    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478370    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478374    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478378    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478382    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478386    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478399    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478404    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478408    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478412    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478416    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478420    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478425    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478429    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478433    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478438    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478442    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478446    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 17:45:05.484402 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478451    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478455    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478460    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478464    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478468    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478472    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478476    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478483    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478488    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478493    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478500    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478504    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478511    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478516    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478521    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 17:45:05.485009 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478526    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.478534    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478737    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478747    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478752    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478756    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478760    2582 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478765    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478769    2582 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478773    2582 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478777    2582 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478781    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478784    2582 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478788    2582 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478792    2582 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 17:45:05.485422 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478796    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478802    2582 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478808    2582 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478812    2582 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478817    2582 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478821    2582 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478825    2582 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478829    2582 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478833    2582 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478837    2582 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478841    2582 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478845    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478849    2582 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478853    2582 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478857    2582 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478862    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478866    2582 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478872    2582 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478877    2582 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 17:45:05.485884 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478881    2582 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478886    2582 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478890    2582 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478894    2582 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478899    2582 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478903    2582 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478908    2582 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478914    2582 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478920    2582 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478925    2582 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478930    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478934    2582 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478939    2582 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478943    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478947    2582 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478952    2582 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478956    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478960    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478965    2582 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 17:45:05.486476 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478969    2582 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478973    2582 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478977    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478982    2582 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478986    2582 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478991    2582 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.478995    2582 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479000    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479004    2582 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479009    2582 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479014    2582 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479019    2582 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479023    2582 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479027    2582 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479032    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479036    2582 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479040    2582 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479044    2582 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479048    2582 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 17:45:05.487337 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479052    2582 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479057    2582 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479062    2582 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479066    2582 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479071    2582 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479075    2582 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479079    2582 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479083    2582 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479087    2582 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479091    2582 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479095    2582 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479101    2582 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479105    2582 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479109    2582 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479113    2582 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479117    2582 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479121    2582 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479125    2582 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479130    2582 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479134    2582 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 17:45:05.487974 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479139    2582 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479143    2582 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479148    2582 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479153    2582 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479158    2582 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479162    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479167    2582 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479171    2582 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:05.479175    2582 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.479183    2582 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.479379    2582 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.482237    2582 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.483808    2582 server.go:1019] "Starting client certificate rotation"
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.483924    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 17:45:05.488529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.483969    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 17:45:05.488936 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.487018    2582 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 17:45:05.490958 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.490936    2582 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 17:45:05.503792 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.503767    2582 log.go:25] "Validated CRI v1 runtime API"
Jul 23 17:45:05.508841 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.508820    2582 log.go:25] "Validated CRI v1 image API"
Jul 23 17:45:05.509765 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.509746    2582 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 17:45:05.516292 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.516267    2582 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9d5aa7e8-9748-46ad-b0e5-2223a7b167af:/dev/nvme0n1p3 e8d6500a-59c8-4f5f-92fe-f4cffc5b65fe:/dev/nvme0n1p4]
Jul 23 17:45:05.516384 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.516290    2582 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 17:45:05.519862 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.519841    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 17:45:05.522363 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.522241    2582 manager.go:216] Machine: {Timestamp:2026-07-23 17:45:05.520720487 +0000 UTC m=+0.360228432 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098813 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23057d9cd01e384a2cfdd8a9274a96 SystemUUID:ec23057d-9cd0-1e38-4a2c-fdd8a9274a96 BootID:e5c05d2a-a0c2-41f1-8773-491fc580fde6 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e1:9e:26:e1:c3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e1:9e:26:e1:c3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:a7:e3:16:26:ba Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 17:45:05.522363 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.522355    2582 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 17:45:05.522516 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.522502    2582 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 17:45:05.526758 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.526725    2582 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 17:45:05.526941 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.526762    2582 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-151.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 17:45:05.528165 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.528152    2582 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 17:45:05.528201 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.528167    2582 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 17:45:05.528201 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.528176    2582 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 17:45:05.528919 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.528895    2582 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 17:45:05.530343 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.530330    2582 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 17:45:05.533950 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.533933    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 17:45:05.534099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534089    2582 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 17:45:05.534142 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534122    2582 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 17:45:05.534142 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534134    2582 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 17:45:05.534207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534151    2582 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 17:45:05.534207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534163    2582 kubelet.go:398] "Adding apiserver pod source"
Jul 23 17:45:05.534207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534173    2582 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 17:45:05.534602 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534589    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 17:45:05.534640 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.534610    2582 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 17:45:05.538187 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.538167    2582 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 17:45:05.540720 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.540703    2582 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 17:45:05.540812 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.540737    2582 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 17:45:05.541049 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.541029    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hzjmq"
Jul 23 17:45:05.542012 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.541996    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542018    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542026    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542032    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542042    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542047    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542070    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542077    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 17:45:05.542082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542084    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 17:45:05.542438 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542090    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 17:45:05.542438 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542106    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 17:45:05.542438 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.542111    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 17:45:05.542680 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.542657    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-151.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 17:45:05.542820 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.542790    2582 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 17:45:05.543885 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.543872    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 17:45:05.543885 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.543885    2582 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 17:45:05.547543 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.547522    2582 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-151.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 17:45:05.547952 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.547938    2582 server.go:1268] "Started kubelet"
Jul 23 17:45:05.548056 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.548023    2582 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 17:45:05.548153 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.548105    2582 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 17:45:05.548213 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.548196    2582 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 17:45:05.548962 ip-10-0-139-151 systemd[1]: Started Kubernetes Kubelet.
Jul 23 17:45:05.549975 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.549924    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hzjmq"
Jul 23 17:45:05.554613 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.554581    2582 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 17:45:05.555704 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.555681    2582 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 17:45:05.556015 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.555945    2582 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 17:45:05.559399 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.557945    2582 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-151.ec2.internal.18c4fc4053193626  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-151.ec2.internal,UID:ip-10-0-139-151.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-151.ec2.internal,},FirstTimestamp:2026-07-23 17:45:05.547826726 +0000 UTC m=+0.387334728,LastTimestamp:2026-07-23 17:45:05.547826726 +0000 UTC m=+0.387334728,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-151.ec2.internal,}"
Jul 23 17:45:05.560898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.560877    2582 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 17:45:05.560998 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.560901    2582 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 17:45:05.561057 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.560996    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:05.561525 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.561498    2582 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 17:45:05.561647 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.561509    2582 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 17:45:05.561751 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.561739    2582 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 17:45:05.561940 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.561919    2582 factory.go:55] Registering systemd factory
Jul 23 17:45:05.562007 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.561943    2582 factory.go:223] Registration of the systemd container factory successfully
Jul 23 17:45:05.562179 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562157    2582 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 17:45:05.562179 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562178    2582 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 17:45:05.562334 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562190    2582 factory.go:153] Registering CRI-O factory
Jul 23 17:45:05.562334 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562209    2582 factory.go:223] Registration of the crio container factory successfully
Jul 23 17:45:05.562334 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562270    2582 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 17:45:05.562334 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562296    2582 factory.go:103] Registering Raw factory
Jul 23 17:45:05.562334 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562318    2582 manager.go:1196] Started watching for new ooms in manager
Jul 23 17:45:05.562795 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.562762    2582 manager.go:349] Starting recovery of all containers
Jul 23 17:45:05.564591 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.564567    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 17:45:05.568263 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.568236    2582 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-151.ec2.internal\" not found" node="ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.573908 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.573878    2582 manager.go:354] Recovery completed
Jul 23 17:45:05.578638 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.578613    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 17:45:05.580804 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.580768    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 17:45:05.580804 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.580800    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 17:45:05.580933 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.580813    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientPID"
Jul 23 17:45:05.581423 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.581411    2582 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 17:45:05.581479 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.581422    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 17:45:05.581479 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.581422    2582 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 17:45:05.581479 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.581464    2582 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 17:45:05.582233 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.582215    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 17:45:05.582302 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.582240    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 17:45:05.582302 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.582256    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientPID"
Jul 23 17:45:05.582780 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.582768    2582 policy_none.go:49] "None policy: Start"
Jul 23 17:45:05.582825 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.582785    2582 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 17:45:05.582825 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.582797    2582 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 17:45:05.583772 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.583762    2582 policy_none.go:47] "Start"
Jul 23 17:45:05.625773 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.625549    2582 manager.go:337] "Starting Device Plugin manager"
Jul 23 17:45:05.625926 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.625835    2582 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 17:45:05.625926 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.625852    2582 server.go:87] "Starting device plugin registration server"
Jul 23 17:45:05.626137 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.626126    2582 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 17:45:05.626173 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.626142    2582 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 17:45:05.626397 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.626245    2582 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 17:45:05.626397 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.626349    2582 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 17:45:05.626397 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.626359    2582 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 17:45:05.627039 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.627018    2582 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 17:45:05.627135 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.627068    2582 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:05.729755 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.729678    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 17:45:05.731006 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.730812    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 17:45:05.731006 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.730846    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 17:45:05.731006 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.730856    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientPID"
Jul 23 17:45:05.731006 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.730885    2582 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.740680 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.740654    2582 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.740804 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.740686    2582 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-151.ec2.internal\": node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:05.754873 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.754846    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:05.776360 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.776320    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 17:45:05.777682 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.777658    2582 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 17:45:05.777883 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.777704    2582 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 17:45:05.777883 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.777733    2582 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 17:45:05.777883 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.777768    2582 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 17:45:05.784065 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.784038    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 17:45:05.858722 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.858674    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:05.878639 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.878600    2582 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal"]
Jul 23 17:45:05.878737 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.878688    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 17:45:05.880447 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.880430    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 17:45:05.880534 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.880462    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 17:45:05.880534 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.880478    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientPID"
Jul 23 17:45:05.881943 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.881931    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 17:45:05.882067 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882051    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.882122 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882082    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 17:45:05.882753 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882737    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 17:45:05.882835 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882763    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 17:45:05.882835 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882775    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientPID"
Jul 23 17:45:05.882835 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882724    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 17:45:05.882940 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882845    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 17:45:05.882940 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.882862    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientPID"
Jul 23 17:45:05.884298 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.884283    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.884352 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.884309    2582 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 17:45:05.885125 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.885106    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 17:45:05.885232 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.885140    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 17:45:05.885232 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.885157    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeHasSufficientPID"
Jul 23 17:45:05.908868 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.908835    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-151.ec2.internal\" not found" node="ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.913764 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.913746    2582 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-151.ec2.internal\" not found" node="ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.959637 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:05.959610    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:05.965428 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.965404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d1a6f68d7fb21bfef8af77aa3c920bb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal\" (UID: \"0d1a6f68d7fb21bfef8af77aa3c920bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.965493 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.965434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d1a6f68d7fb21bfef8af77aa3c920bb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal\" (UID: \"0d1a6f68d7fb21bfef8af77aa3c920bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:05.965493 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:05.965454    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/845580dd794356674919cca719ca31b8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-151.ec2.internal\" (UID: \"845580dd794356674919cca719ca31b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.061367 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.061327    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.065861 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.065842    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d1a6f68d7fb21bfef8af77aa3c920bb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal\" (UID: \"0d1a6f68d7fb21bfef8af77aa3c920bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.065927 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.065889    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d1a6f68d7fb21bfef8af77aa3c920bb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal\" (UID: \"0d1a6f68d7fb21bfef8af77aa3c920bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.065927 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.065915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/845580dd794356674919cca719ca31b8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-151.ec2.internal\" (UID: \"845580dd794356674919cca719ca31b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.066018 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.065925    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0d1a6f68d7fb21bfef8af77aa3c920bb-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal\" (UID: \"0d1a6f68d7fb21bfef8af77aa3c920bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.066018 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.065960    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d1a6f68d7fb21bfef8af77aa3c920bb-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal\" (UID: \"0d1a6f68d7fb21bfef8af77aa3c920bb\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.066018 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.065959    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/845580dd794356674919cca719ca31b8-config\") pod \"kube-apiserver-proxy-ip-10-0-139-151.ec2.internal\" (UID: \"845580dd794356674919cca719ca31b8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.168516 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.168468    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.214495 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.214449    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.217289 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.217271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:06.269325 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.269284    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.370456 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.370378    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.471464 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.471432    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.483476 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.483447    2582 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 17:45:06.483633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.483615    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 17:45:06.483674 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.483655    2582 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 17:45:06.559057 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.559003    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 17:40:06 +0000 UTC" deadline="2028-02-14 04:43:18.668048074 +0000 UTC"
Jul 23 17:45:06.559057 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.559047    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13690h58m12.109004647s"
Jul 23 17:45:06.572026 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.571995    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.616069 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.616043    2582 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 17:45:06.673098 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.673066    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.755335 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:06.755294    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod845580dd794356674919cca719ca31b8.slice/crio-fd51ddc20ba3a24ca6f95f327443d64ea259ad6a0b0d5e772f6d776f55d3b616 WatchSource:0}: Error finding container fd51ddc20ba3a24ca6f95f327443d64ea259ad6a0b0d5e772f6d776f55d3b616: Status 404 returned error can't find the container with id fd51ddc20ba3a24ca6f95f327443d64ea259ad6a0b0d5e772f6d776f55d3b616
Jul 23 17:45:06.755594 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:06.755571    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d1a6f68d7fb21bfef8af77aa3c920bb.slice/crio-53e826cf670d97eae191d498ee202afe96fa4c44e06882360880d6f9a138cfd0 WatchSource:0}: Error finding container 53e826cf670d97eae191d498ee202afe96fa4c44e06882360880d6f9a138cfd0: Status 404 returned error can't find the container with id 53e826cf670d97eae191d498ee202afe96fa4c44e06882360880d6f9a138cfd0
Jul 23 17:45:06.761501 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.761483    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 17:45:06.773682 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.773659    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.782060 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.782009    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal" event={"ID":"0d1a6f68d7fb21bfef8af77aa3c920bb","Type":"ContainerStarted","Data":"53e826cf670d97eae191d498ee202afe96fa4c44e06882360880d6f9a138cfd0"}
Jul 23 17:45:06.783100 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:06.783076    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal" event={"ID":"845580dd794356674919cca719ca31b8","Type":"ContainerStarted","Data":"fd51ddc20ba3a24ca6f95f327443d64ea259ad6a0b0d5e772f6d776f55d3b616"}
Jul 23 17:45:06.874715 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.874673    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:06.975886 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:06.975792    2582 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-151.ec2.internal\" not found"
Jul 23 17:45:07.073599 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.073563    2582 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 17:45:07.166853 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.166814    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:07.180363 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.180328    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 17:45:07.181599 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.181578    2582 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal"
Jul 23 17:45:07.190357 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.190325    2582 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 17:45:07.398408 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.398374    2582 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 17:45:07.537059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.537026    2582 apiserver.go:52] "Watching apiserver"
Jul 23 17:45:07.546753 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.546692    2582 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 17:45:07.549452 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.549421    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-xn4ct","kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal","openshift-dns/node-resolver-l79t9","openshift-image-registry/node-ca-rb9ft","openshift-network-diagnostics/network-check-target-mzb6t","openshift-network-operator/iptables-alerter-nb5kz","kube-system/global-pull-secret-syncer-tcqbt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9","openshift-cluster-node-tuning-operator/tuned-94rjj","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal","openshift-multus/multus-additional-cni-plugins-76gc6","openshift-multus/multus-rcf5c","openshift-multus/network-metrics-daemon-vfv7s","openshift-ovn-kubernetes/ovnkube-node-pnrc8"]
Jul 23 17:45:07.551412 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.551388    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.551512 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.551472    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:07.553457 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.553433    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:07.553560 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.553540    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:07.554592 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.554572    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.555791 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.555757    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.557573 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.557265    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:07.557573 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.557349    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:07.557573 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.557494    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5zsgj\""
Jul 23 17:45:07.557838 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.557572    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 17:45:07.557838 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.557599    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 17:45:07.558560 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.558353    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 17:45:07.558697 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.558680    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mcrkd\""
Jul 23 17:45:07.558757 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.558703    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 17:45:07.558821 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.558789    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 17:45:07.558880 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.558719    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.560034 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.560015    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.561272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.561254    2582 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 17:45:07.562010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.561390    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.562010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.561574    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 17:45:07.562010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.561704    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-lsk7x\""
Jul 23 17:45:07.562010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.561835    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 17:45:07.562010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.562005    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 17:45:07.562542 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.562531    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 17:45:07.562929 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.562898    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 17:45:07.563019 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.562962    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-f642k\""
Jul 23 17:45:07.563618 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.563577    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.564334 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.564310    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-kzq6q\""
Jul 23 17:45:07.564443 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.564312    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 17:45:07.564443 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.564420    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 17:45:07.565279 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.565261    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.565934 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.565916    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 17:45:07.566194 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.566179    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 17:45:07.566272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.566181    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 17:45:07.566626 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.566607    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 17:45:07.566713 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.566630    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kk7cv\""
Jul 23 17:45:07.567102 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.567083    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.567597 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.567542    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 17:45:07.567877 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.567855    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 17:45:07.567999 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.567976    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 17:45:07.568101 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.568052    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 17:45:07.568199 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.568179    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 17:45:07.568342 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.568323    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-bqp2q\""
Jul 23 17:45:07.568888 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.568869    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.570604 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.570576    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 17:45:07.570703 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.570576    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 17:45:07.571370 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.570981    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 17:45:07.571370 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.571053    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 17:45:07.571370 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.571119    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-tc9gt\""
Jul 23 17:45:07.571625 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.571414    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 17:45:07.571625 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.571461    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-96q58\""
Jul 23 17:45:07.571625 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.571420    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 17:45:07.578331 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.577924    2582 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 17:45:07.579538 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579511    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-lib-modules\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.579648 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579564    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d602a-92db-4f5e-86a3-7f25c4209c6e-konnectivity-ca\") pod \"konnectivity-agent-xn4ct\" (UID: \"ed8d602a-92db-4f5e-86a3-7f25c4209c6e\") " pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.579648 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579602    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.579648 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579631    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/26784fee-febb-4712-9906-14c5a6d9a352-hosts-file\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.579819 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7s7b\" (UniqueName: \"kubernetes.io/projected/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-kube-api-access-f7s7b\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.579819 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579711    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysconfig\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.579819 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579762    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-kubernetes\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.579979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579817    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-systemd-units\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.579979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dde8d19a-bea3-48da-9fee-a5353c5e385e-dbus\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.579979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-iptables-alerter-script\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.579979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-systemd\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.579979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579961    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed8d602a-92db-4f5e-86a3-7f25c4209c6e-agent-certs\") pod \"konnectivity-agent-xn4ct\" (UID: \"ed8d602a-92db-4f5e-86a3-7f25c4209c6e\") " pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.579993    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-etc-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580026    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovnkube-config\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580053    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pwcd\" (UniqueName: \"kubernetes.io/projected/26784fee-febb-4712-9906-14c5a6d9a352-kube-api-access-4pwcd\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580090    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-host-slash\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580123    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69f296a4-0368-4e06-b923-fa7368469182-tmp\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580161    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580197    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dde8d19a-bea3-48da-9fee-a5353c5e385e-kubelet-config\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.580243 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580224    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-systemd\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580595 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580255    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24b62c73-baa3-4159-9529-58aa59ebf21d-serviceca\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.580595 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580289    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj9d6\" (UniqueName: \"kubernetes.io/projected/24b62c73-baa3-4159-9529-58aa59ebf21d-kube-api-access-vj9d6\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.580595 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580322    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-sys\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.580595 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580381    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmwbd\" (UniqueName: \"kubernetes.io/projected/69f296a4-0368-4e06-b923-fa7368469182-kube-api-access-hmwbd\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.580595 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580408    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-log-socket\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580595 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-cni-bin\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580595 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-cni-netd\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580583    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysctl-conf\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.580924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580639    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-host\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.580924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580685    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69f296a4-0368-4e06-b923-fa7368469182-etc-tuned\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.580924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580714    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-slash\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580776    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-run-netns\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580854    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-var-lib-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.580924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580903    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-node-log\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.581241 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580942    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovn-node-metrics-cert\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.581241 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.580970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-run\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.581241 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581028    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:07.581393 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581242    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-modprobe-d\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.581393 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581338    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-kubelet\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.581753 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581709    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.581898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-ovn\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.581898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581795    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.581898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581820    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-env-overrides\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.581898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581845    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24b62c73-baa3-4159-9529-58aa59ebf21d-host\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.581898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysctl-d\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.581898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581882    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-var-lib-kubelet\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.582180 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581901    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovnkube-script-lib\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.582180 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581918    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkzcs\" (UniqueName: \"kubernetes.io/projected/aa5549a4-15ae-44dd-9c68-e46430112f3c-kube-api-access-lkzcs\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.582180 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581935    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:07.582180 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4tp5\" (UniqueName: \"kubernetes.io/projected/59bb77c5-cb7e-41f8-81dc-aef0841f958f-kube-api-access-z4tp5\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:07.582180 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.581967    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26784fee-febb-4712-9906-14c5a6d9a352-tmp-dir\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.602625 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.602596    2582 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-gv7kz"
Jul 23 17:45:07.614188 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.614157    2582 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gv7kz"
Jul 23 17:45:07.662629 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.662539    2582 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 17:45:07.683001 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.682959    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovn-node-metrics-cert\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683001 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovnkube-script-lib\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4tp5\" (UniqueName: \"kubernetes.io/projected/59bb77c5-cb7e-41f8-81dc-aef0841f958f-kube-api-access-z4tp5\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24b62c73-baa3-4159-9529-58aa59ebf21d-serviceca\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683057    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj9d6\" (UniqueName: \"kubernetes.io/projected/24b62c73-baa3-4159-9529-58aa59ebf21d-kube-api-access-vj9d6\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-cni-bin\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683107    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv825\" (UniqueName: \"kubernetes.io/projected/ec002160-229b-4bc8-a52f-12354c87c023-kube-api-access-sv825\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683166    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-socket-dir-parent\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683193    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysctl-conf\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69f296a4-0368-4e06-b923-fa7368469182-etc-tuned\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.683265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683244    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-cni-bin\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683305    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-run-netns\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683247    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-run-netns\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-var-lib-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/26784fee-febb-4712-9906-14c5a6d9a352-hosts-file\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683412    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-os-release\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683423    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-var-lib-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683437    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-netns\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysctl-conf\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683481    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-multus-certs\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683489    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/26784fee-febb-4712-9906-14c5a6d9a352-hosts-file\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683516    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-run\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683515    2582 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683549    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683577    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-modprobe-d\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683689    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-modprobe-d\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.683728 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683699    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-run\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/24b62c73-baa3-4159-9529-58aa59ebf21d-serviceca\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-ovn\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683867    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lkzcs\" (UniqueName: \"kubernetes.io/projected/aa5549a4-15ae-44dd-9c68-e46430112f3c-kube-api-access-lkzcs\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683874    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-ovn\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683908    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683908    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovnkube-script-lib\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683938    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-system-cni-dir\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-cni-binary-copy\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.683995    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24b62c73-baa3-4159-9529-58aa59ebf21d-host\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.684009    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684019    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26784fee-febb-4712-9906-14c5a6d9a352-tmp-dir\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684046    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7s7b\" (UniqueName: \"kubernetes.io/projected/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-kube-api-access-f7s7b\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684062    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/24b62c73-baa3-4159-9529-58aa59ebf21d-host\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.684080    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs podName:59bb77c5-cb7e-41f8-81dc-aef0841f958f nodeName:}" failed. No retries permitted until 2026-07-23 17:45:08.18406154 +0000 UTC m=+3.023569467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs") pod "network-metrics-daemon-vfv7s" (UID: "59bb77c5-cb7e-41f8-81dc-aef0841f958f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684102    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-socket-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684133    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-device-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.684507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684160    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-cnibin\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684187    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-cni-binary-copy\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684219    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-k8s-cni-cncf-io\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-lib-modules\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684276    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-cnibin\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-conf-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684328    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/26784fee-febb-4712-9906-14c5a6d9a352-tmp-dir\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684330    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-etc-kubernetes\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684372    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-lib-modules\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684380    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bw2vs\" (UniqueName: \"kubernetes.io/projected/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-kube-api-access-bw2vs\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-systemd-units\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684437    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-iptables-alerter-script\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684464    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-tuning-conf-dir\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684475    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-systemd-units\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684495    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-host-slash\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69f296a4-0368-4e06-b923-fa7368469182-tmp\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684547    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.685364 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684556    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-host-slash\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684576    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dde8d19a-bea3-48da-9fee-a5353c5e385e-kubelet-config\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684602    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-systemd\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684627    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-sys\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684629    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684651    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmwbd\" (UniqueName: \"kubernetes.io/projected/69f296a4-0368-4e06-b923-fa7368469182-kube-api-access-hmwbd\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684678    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-log-socket\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684685    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-systemd\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684704    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-cni-netd\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684763    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/dde8d19a-bea3-48da-9fee-a5353c5e385e-kubelet-config\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684820    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-cni-netd\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684842    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-log-socket\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684884    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-sys\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684884    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-registration-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-sys-fs\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.684954    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-cni-bin\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-host\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.686217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685028    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-iptables-alerter-script\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685068    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-slash\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685035    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-slash\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-node-log\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685150    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-node-log\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-cni-multus\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685112    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-host\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685189    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-hostroot\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-kubelet\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685283    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685335    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685361    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-env-overrides\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685368    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-kubelet\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685418    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-run-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685424    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-host-run-ovn-kubernetes\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685465    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-cni-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685498    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysctl-d\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-var-lib-kubelet\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.687099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685577    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-var-lib-kubelet\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685607    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-os-release\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685638    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d602a-92db-4f5e-86a3-7f25c4209c6e-konnectivity-ca\") pod \"konnectivity-agent-xn4ct\" (UID: \"ed8d602a-92db-4f5e-86a3-7f25c4209c6e\") " pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685628    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysctl-d\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685666    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685693    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685763    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-kubelet\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685797    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysconfig\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685823    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-kubernetes\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685849    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dde8d19a-bea3-48da-9fee-a5353c5e385e-dbus\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685877    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685903    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685951    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-system-cni-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685977    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-systemd\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.685995    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-env-overrides\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686003    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed8d602a-92db-4f5e-86a3-7f25c4209c6e-agent-certs\") pod \"konnectivity-agent-xn4ct\" (UID: \"ed8d602a-92db-4f5e-86a3-7f25c4209c6e\") " pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-etc-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.688047 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686112    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovnkube-config\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pwcd\" (UniqueName: \"kubernetes.io/projected/26784fee-febb-4712-9906-14c5a6d9a352-kube-api-access-4pwcd\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686214    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686235    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d602a-92db-4f5e-86a3-7f25c4209c6e-konnectivity-ca\") pod \"konnectivity-agent-xn4ct\" (UID: \"ed8d602a-92db-4f5e-86a3-7f25c4209c6e\") " pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.686246    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686264    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dw5xk\" (UniqueName: \"kubernetes.io/projected/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-kube-api-access-dw5xk\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.686299    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret podName:dde8d19a-bea3-48da-9fee-a5353c5e385e nodeName:}" failed. No retries permitted until 2026-07-23 17:45:08.186282163 +0000 UTC m=+3.025790104 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret") pod "global-pull-secret-syncer-tcqbt" (UID: "dde8d19a-bea3-48da-9fee-a5353c5e385e") : object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686316    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/aa5549a4-15ae-44dd-9c68-e46430112f3c-etc-openvswitch\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686320    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-daemon-config\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686441    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-sysconfig\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686495    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-kubernetes\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686601    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/dde8d19a-bea3-48da-9fee-a5353c5e385e-dbus\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686868    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovnkube-config\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.686988    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/69f296a4-0368-4e06-b923-fa7368469182-etc-systemd\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.687765    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/69f296a4-0368-4e06-b923-fa7368469182-etc-tuned\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.687824    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/69f296a4-0368-4e06-b923-fa7368469182-tmp\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.688965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.687903    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/aa5549a4-15ae-44dd-9c68-e46430112f3c-ovn-node-metrics-cert\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.689924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.689902    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/ed8d602a-92db-4f5e-86a3-7f25c4209c6e-agent-certs\") pod \"konnectivity-agent-xn4ct\" (UID: \"ed8d602a-92db-4f5e-86a3-7f25c4209c6e\") " pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.694464 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.694410    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 17:45:07.695072 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.695052    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 17:45:07.695173 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.695077    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5vzb7 for pod openshift-network-diagnostics/network-check-target-mzb6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:07.695980 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:07.695957    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7 podName:e0f08cf4-479e-4017-9776-98f34bd6fef3 nodeName:}" failed. No retries permitted until 2026-07-23 17:45:08.195931422 +0000 UTC m=+3.035439377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5vzb7" (UniqueName: "kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7") pod "network-check-target-mzb6t" (UID: "e0f08cf4-479e-4017-9776-98f34bd6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:07.697222 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.697147    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkzcs\" (UniqueName: \"kubernetes.io/projected/aa5549a4-15ae-44dd-9c68-e46430112f3c-kube-api-access-lkzcs\") pod \"ovnkube-node-pnrc8\" (UID: \"aa5549a4-15ae-44dd-9c68-e46430112f3c\") " pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.697669 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.697644    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj9d6\" (UniqueName: \"kubernetes.io/projected/24b62c73-baa3-4159-9529-58aa59ebf21d-kube-api-access-vj9d6\") pod \"node-ca-rb9ft\" (UID: \"24b62c73-baa3-4159-9529-58aa59ebf21d\") " pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.698330 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.698304    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmwbd\" (UniqueName: \"kubernetes.io/projected/69f296a4-0368-4e06-b923-fa7368469182-kube-api-access-hmwbd\") pod \"tuned-94rjj\" (UID: \"69f296a4-0368-4e06-b923-fa7368469182\") " pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.698504 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.698481    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4tp5\" (UniqueName: \"kubernetes.io/projected/59bb77c5-cb7e-41f8-81dc-aef0841f958f-kube-api-access-z4tp5\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:07.699190 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.699147    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7s7b\" (UniqueName: \"kubernetes.io/projected/ebfdf19f-561d-4ee7-838f-1135e9e2df8e-kube-api-access-f7s7b\") pod \"iptables-alerter-nb5kz\" (UID: \"ebfdf19f-561d-4ee7-838f-1135e9e2df8e\") " pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.699296 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.699277    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pwcd\" (UniqueName: \"kubernetes.io/projected/26784fee-febb-4712-9906-14c5a6d9a352-kube-api-access-4pwcd\") pod \"node-resolver-l79t9\" (UID: \"26784fee-febb-4712-9906-14c5a6d9a352\") " pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.786857 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.786816    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv825\" (UniqueName: \"kubernetes.io/projected/ec002160-229b-4bc8-a52f-12354c87c023-kube-api-access-sv825\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.786867    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-socket-dir-parent\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.786901    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-os-release\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.786927    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-netns\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.786952    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-multus-certs\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787004    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-system-cni-dir\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787026    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-cni-binary-copy\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-socket-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.787071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787053    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-os-release\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787077    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-device-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787108    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-cnibin\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787129    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-cni-binary-copy\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787150    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-k8s-cni-cncf-io\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787172    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-cnibin\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787183    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-multus-certs\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787194    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-conf-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-etc-kubernetes\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787245    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-conf-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787256    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bw2vs\" (UniqueName: \"kubernetes.io/projected/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-kube-api-access-bw2vs\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-tuning-conf-dir\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787324    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-registration-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787354    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-sys-fs\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-cni-bin\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787389    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-socket-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787408    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-cni-multus\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.787505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-hostroot\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787447    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-device-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-cni-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787488    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-cnibin\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787492    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-os-release\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787542    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787570    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-kubelet\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787643    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787669    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-system-cni-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dw5xk\" (UniqueName: \"kubernetes.io/projected/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-kube-api-access-dw5xk\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787748    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-daemon-config\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787901    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-cni-binary-copy\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787962    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-system-cni-dir\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787964    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-k8s-cni-cncf-io\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.787127    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-socket-dir-parent\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.788399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788017    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-cnibin\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788052    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-hostroot\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788066    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-run-netns\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788115    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-cni-bin\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788115    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-cni-multus\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788161    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-tuning-conf-dir\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788184    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-cni-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788189    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ec002160-229b-4bc8-a52f-12354c87c023-os-release\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788228    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-etc-kubernetes\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788234    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-registration-dir\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788266    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-sys-fs\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788281    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-multus-daemon-config\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788307    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-host-var-lib-kubelet\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788343    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-etc-selinux\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-system-cni-dir\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788735    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.789266 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788767    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.790144 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.788890    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ec002160-229b-4bc8-a52f-12354c87c023-cni-binary-copy\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.796798 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.796768    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv825\" (UniqueName: \"kubernetes.io/projected/ec002160-229b-4bc8-a52f-12354c87c023-kube-api-access-sv825\") pod \"multus-additional-cni-plugins-76gc6\" (UID: \"ec002160-229b-4bc8-a52f-12354c87c023\") " pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.798131 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.798091    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dw5xk\" (UniqueName: \"kubernetes.io/projected/a672b3c6-b7ef-4f54-b17b-6586ee8d7868-kube-api-access-dw5xk\") pod \"aws-ebs-csi-driver-node-8lcv9\" (UID: \"a672b3c6-b7ef-4f54-b17b-6586ee8d7868\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.798265 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.798239    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bw2vs\" (UniqueName: \"kubernetes.io/projected/88012f00-e808-4fe4-9fbe-f0825cfd0ff7-kube-api-access-bw2vs\") pod \"multus-rcf5c\" (UID: \"88012f00-e808-4fe4-9fbe-f0825cfd0ff7\") " pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.865926 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.865884    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-l79t9"
Jul 23 17:45:07.874006 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.873978    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rb9ft"
Jul 23 17:45:07.884368 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.884339    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-nb5kz"
Jul 23 17:45:07.890639 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.890609    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-94rjj"
Jul 23 17:45:07.897644 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.897613    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:07.904538 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.904509    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:07.911377 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.911353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9"
Jul 23 17:45:07.919134 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.918977    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-76gc6"
Jul 23 17:45:07.919357 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.919326    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8d602a_92db_4f5e_86a3_7f25c4209c6e.slice/crio-31a31e61972ed8f7101f779ecc18c26546c4ffdf38b43ef21483a115806d1620 WatchSource:0}: Error finding container 31a31e61972ed8f7101f779ecc18c26546c4ffdf38b43ef21483a115806d1620: Status 404 returned error can't find the container with id 31a31e61972ed8f7101f779ecc18c26546c4ffdf38b43ef21483a115806d1620
Jul 23 17:45:07.923299 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.923276    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfdf19f_561d_4ee7_838f_1135e9e2df8e.slice/crio-8cc331b57810516f60b422c36478f41f1af97a212136b97e24260910f6d87ac8 WatchSource:0}: Error finding container 8cc331b57810516f60b422c36478f41f1af97a212136b97e24260910f6d87ac8: Status 404 returned error can't find the container with id 8cc331b57810516f60b422c36478f41f1af97a212136b97e24260910f6d87ac8
Jul 23 17:45:07.924054 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:07.924029    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rcf5c"
Jul 23 17:45:07.924134 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.924081    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69f296a4_0368_4e06_b923_fa7368469182.slice/crio-790576bf0ab8a6e58f5ced2d09371ee1f20bb6bf5a961f7c4347d8c46e1786fd WatchSource:0}: Error finding container 790576bf0ab8a6e58f5ced2d09371ee1f20bb6bf5a961f7c4347d8c46e1786fd: Status 404 returned error can't find the container with id 790576bf0ab8a6e58f5ced2d09371ee1f20bb6bf5a961f7c4347d8c46e1786fd
Jul 23 17:45:07.925080 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.925037    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa5549a4_15ae_44dd_9c68_e46430112f3c.slice/crio-dc2a63f698e7153b18125b1a427b04aeec1a59998e50618596e42247d7eeeb8d WatchSource:0}: Error finding container dc2a63f698e7153b18125b1a427b04aeec1a59998e50618596e42247d7eeeb8d: Status 404 returned error can't find the container with id dc2a63f698e7153b18125b1a427b04aeec1a59998e50618596e42247d7eeeb8d
Jul 23 17:45:07.927489 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.926562    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24b62c73_baa3_4159_9529_58aa59ebf21d.slice/crio-dd8655b036b9616c79c709ab77ac00ad5adc6afded3f5c07d16816177b156c2d WatchSource:0}: Error finding container dd8655b036b9616c79c709ab77ac00ad5adc6afded3f5c07d16816177b156c2d: Status 404 returned error can't find the container with id dd8655b036b9616c79c709ab77ac00ad5adc6afded3f5c07d16816177b156c2d
Jul 23 17:45:07.932530 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.932500    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda672b3c6_b7ef_4f54_b17b_6586ee8d7868.slice/crio-a0a4f954ba2aa1b892acd9e21af47a744ecdd85331060b079d5dd7a4ceaf01fd WatchSource:0}: Error finding container a0a4f954ba2aa1b892acd9e21af47a744ecdd85331060b079d5dd7a4ceaf01fd: Status 404 returned error can't find the container with id a0a4f954ba2aa1b892acd9e21af47a744ecdd85331060b079d5dd7a4ceaf01fd
Jul 23 17:45:07.936448 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.936425    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec002160_229b_4bc8_a52f_12354c87c023.slice/crio-72eb0a4119e379e854be1bd453ec6d7cfea22e366cc06ad861b6e5acefc3d314 WatchSource:0}: Error finding container 72eb0a4119e379e854be1bd453ec6d7cfea22e366cc06ad861b6e5acefc3d314: Status 404 returned error can't find the container with id 72eb0a4119e379e854be1bd453ec6d7cfea22e366cc06ad861b6e5acefc3d314
Jul 23 17:45:07.937262 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:07.937231    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88012f00_e808_4fe4_9fbe_f0825cfd0ff7.slice/crio-d0b17d28aadfb079c6774bc0af7dd349f3b7b0bf7077d96b7fd1a5018a311c5c WatchSource:0}: Error finding container d0b17d28aadfb079c6774bc0af7dd349f3b7b0bf7077d96b7fd1a5018a311c5c: Status 404 returned error can't find the container with id d0b17d28aadfb079c6774bc0af7dd349f3b7b0bf7077d96b7fd1a5018a311c5c
Jul 23 17:45:08.047736 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.047681    2582 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 17:45:08.192178 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.192071    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:08.192178 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.192141    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:08.192398 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.192249    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:08.192398 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.192257    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:08.192398 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.192326    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret podName:dde8d19a-bea3-48da-9fee-a5353c5e385e nodeName:}" failed. No retries permitted until 2026-07-23 17:45:09.192310658 +0000 UTC m=+4.031818585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret") pod "global-pull-secret-syncer-tcqbt" (UID: "dde8d19a-bea3-48da-9fee-a5353c5e385e") : object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:08.192398 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.192342    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs podName:59bb77c5-cb7e-41f8-81dc-aef0841f958f nodeName:}" failed. No retries permitted until 2026-07-23 17:45:09.192335463 +0000 UTC m=+4.031843390 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs") pod "network-metrics-daemon-vfv7s" (UID: "59bb77c5-cb7e-41f8-81dc-aef0841f958f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:08.292898 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.292861    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:08.293079 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.293043    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 17:45:08.293079 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.293069    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 17:45:08.293079 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.293081    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5vzb7 for pod openshift-network-diagnostics/network-check-target-mzb6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:08.293183 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.293134    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7 podName:e0f08cf4-479e-4017-9776-98f34bd6fef3 nodeName:}" failed. No retries permitted until 2026-07-23 17:45:09.293119843 +0000 UTC m=+4.132627770 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vzb7" (UniqueName: "kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7") pod "network-check-target-mzb6t" (UID: "e0f08cf4-479e-4017-9776-98f34bd6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:08.617923 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.617877    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 17:40:08 +0000 UTC" deadline="2028-05-06 03:01:52.838099202 +0000 UTC"
Jul 23 17:45:08.617923 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.617920    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15657h16m44.220184872s"
Jul 23 17:45:08.778966 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.778925    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:08.779148 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.779078    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:08.779551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.779529    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:08.779671 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:08.779649    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:08.802371 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.802319    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rcf5c" event={"ID":"88012f00-e808-4fe4-9fbe-f0825cfd0ff7","Type":"ContainerStarted","Data":"d0b17d28aadfb079c6774bc0af7dd349f3b7b0bf7077d96b7fd1a5018a311c5c"}
Jul 23 17:45:08.812007 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.811966    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerStarted","Data":"72eb0a4119e379e854be1bd453ec6d7cfea22e366cc06ad861b6e5acefc3d314"}
Jul 23 17:45:08.830523 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.830427    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-94rjj" event={"ID":"69f296a4-0368-4e06-b923-fa7368469182","Type":"ContainerStarted","Data":"790576bf0ab8a6e58f5ced2d09371ee1f20bb6bf5a961f7c4347d8c46e1786fd"}
Jul 23 17:45:08.837162 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.837040    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9" event={"ID":"a672b3c6-b7ef-4f54-b17b-6586ee8d7868","Type":"ContainerStarted","Data":"a0a4f954ba2aa1b892acd9e21af47a744ecdd85331060b079d5dd7a4ceaf01fd"}
Jul 23 17:45:08.841730 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.841646    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l79t9" event={"ID":"26784fee-febb-4712-9906-14c5a6d9a352","Type":"ContainerStarted","Data":"a7e8d60b9453d358d7c0925c2bd0f092aa72eca5c37a1c116a16d2c3e5c47349"}
Jul 23 17:45:08.845602 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.845571    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rb9ft" event={"ID":"24b62c73-baa3-4159-9529-58aa59ebf21d","Type":"ContainerStarted","Data":"dd8655b036b9616c79c709ab77ac00ad5adc6afded3f5c07d16816177b156c2d"}
Jul 23 17:45:08.853147 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.852826    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"dc2a63f698e7153b18125b1a427b04aeec1a59998e50618596e42247d7eeeb8d"}
Jul 23 17:45:08.865161 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.865055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nb5kz" event={"ID":"ebfdf19f-561d-4ee7-838f-1135e9e2df8e","Type":"ContainerStarted","Data":"8cc331b57810516f60b422c36478f41f1af97a212136b97e24260910f6d87ac8"}
Jul 23 17:45:08.875182 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:08.875098    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xn4ct" event={"ID":"ed8d602a-92db-4f5e-86a3-7f25c4209c6e","Type":"ContainerStarted","Data":"31a31e61972ed8f7101f779ecc18c26546c4ffdf38b43ef21483a115806d1620"}
Jul 23 17:45:09.201917 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:09.201824    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:09.202093 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:09.201916    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:09.202093 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.202061    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:09.202211 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.202128    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret podName:dde8d19a-bea3-48da-9fee-a5353c5e385e nodeName:}" failed. No retries permitted until 2026-07-23 17:45:11.202108601 +0000 UTC m=+6.041616532 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret") pod "global-pull-secret-syncer-tcqbt" (UID: "dde8d19a-bea3-48da-9fee-a5353c5e385e") : object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:09.202596 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.202574    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:09.202695 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.202635    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs podName:59bb77c5-cb7e-41f8-81dc-aef0841f958f nodeName:}" failed. No retries permitted until 2026-07-23 17:45:11.202617941 +0000 UTC m=+6.042125875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs") pod "network-metrics-daemon-vfv7s" (UID: "59bb77c5-cb7e-41f8-81dc-aef0841f958f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:09.303294 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:09.303250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:09.303511 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.303489    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 17:45:09.303582 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.303523    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 17:45:09.303582 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.303542    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5vzb7 for pod openshift-network-diagnostics/network-check-target-mzb6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:09.303690 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.303614    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7 podName:e0f08cf4-479e-4017-9776-98f34bd6fef3 nodeName:}" failed. No retries permitted until 2026-07-23 17:45:11.303589647 +0000 UTC m=+6.143097576 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vzb7" (UniqueName: "kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7") pod "network-check-target-mzb6t" (UID: "e0f08cf4-479e-4017-9776-98f34bd6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:09.619088 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:09.619014    2582 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 17:40:08 +0000 UTC" deadline="2028-01-27 20:43:00.073299352 +0000 UTC"
Jul 23 17:45:09.619088 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:09.619085    2582 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13274h57m50.454218976s"
Jul 23 17:45:09.780631 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:09.780095    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:09.780631 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:09.780243    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:10.778531 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:10.778487    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:10.779160 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:10.778652    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:10.779160 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:10.779096    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:10.779301 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:10.779190    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:11.224400 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:11.223969    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:11.224400 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:11.224047    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:11.224400 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.224195    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:11.224400 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.224263    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret podName:dde8d19a-bea3-48da-9fee-a5353c5e385e nodeName:}" failed. No retries permitted until 2026-07-23 17:45:15.224242697 +0000 UTC m=+10.063750625 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret") pod "global-pull-secret-syncer-tcqbt" (UID: "dde8d19a-bea3-48da-9fee-a5353c5e385e") : object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:11.224400 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.224265    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:11.224400 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.224337    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs podName:59bb77c5-cb7e-41f8-81dc-aef0841f958f nodeName:}" failed. No retries permitted until 2026-07-23 17:45:15.224316588 +0000 UTC m=+10.063824520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs") pod "network-metrics-daemon-vfv7s" (UID: "59bb77c5-cb7e-41f8-81dc-aef0841f958f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:11.325196 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:11.325149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:11.325396 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.325346    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 17:45:11.325396 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.325370    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 17:45:11.325396 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.325384    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5vzb7 for pod openshift-network-diagnostics/network-check-target-mzb6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:11.325558 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.325446    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7 podName:e0f08cf4-479e-4017-9776-98f34bd6fef3 nodeName:}" failed. No retries permitted until 2026-07-23 17:45:15.325426566 +0000 UTC m=+10.164934500 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vzb7" (UniqueName: "kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7") pod "network-check-target-mzb6t" (UID: "e0f08cf4-479e-4017-9776-98f34bd6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:11.778539 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:11.778501    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:11.779085 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:11.778648    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:12.779010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:12.778319    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:12.779010 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:12.778458    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:12.779010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:12.778845    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:12.779010 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:12.778969    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:13.779031 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:13.778943    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:13.779497 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:13.779087    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:14.778327 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:14.778286    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:14.778507 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:14.778438    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:14.778550 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:14.778526    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:14.778644 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:14.778612    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:15.260776 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:15.260567    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:15.260776 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:15.260700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:15.260776 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.260727    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:15.261340 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.260810    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret podName:dde8d19a-bea3-48da-9fee-a5353c5e385e nodeName:}" failed. No retries permitted until 2026-07-23 17:45:23.260789462 +0000 UTC m=+18.100297391 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret") pod "global-pull-secret-syncer-tcqbt" (UID: "dde8d19a-bea3-48da-9fee-a5353c5e385e") : object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:15.261340 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.260815    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:15.261340 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.260879    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs podName:59bb77c5-cb7e-41f8-81dc-aef0841f958f nodeName:}" failed. No retries permitted until 2026-07-23 17:45:23.260861434 +0000 UTC m=+18.100369364 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs") pod "network-metrics-daemon-vfv7s" (UID: "59bb77c5-cb7e-41f8-81dc-aef0841f958f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:15.361608 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:15.361424    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:15.361608 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.361592    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 17:45:15.362010 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.361622    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 17:45:15.362010 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.361636    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5vzb7 for pod openshift-network-diagnostics/network-check-target-mzb6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:15.362010 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.361717    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7 podName:e0f08cf4-479e-4017-9776-98f34bd6fef3 nodeName:}" failed. No retries permitted until 2026-07-23 17:45:23.361683513 +0000 UTC m=+18.201191443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vzb7" (UniqueName: "kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7") pod "network-check-target-mzb6t" (UID: "e0f08cf4-479e-4017-9776-98f34bd6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:15.779732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:15.779681    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:15.779903 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:15.779801    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:16.778717 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:16.778668    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:16.779116 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:16.778822    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:16.779160 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:16.779112    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:16.779271 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:16.779246    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:17.778506 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:17.778471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:17.778677 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:17.778610    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:18.778408 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:18.778375    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:18.778827 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:18.778375    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:18.778827 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:18.778499    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:18.778827 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:18.778598    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:19.778138 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:19.778107    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:19.778320 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:19.778259    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:20.778930 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:20.778890    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:20.779358 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:20.778930    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:20.779358 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:20.779020    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:20.779358 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:20.779171    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:21.778510 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:21.778475    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:21.778695 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:21.778608    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:22.778833 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:22.778796    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:22.779295 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:22.778797    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:22.779295 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:22.778934    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:22.779295 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:22.778982    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:23.323764 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:23.323693    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:23.323965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:23.323789    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:23.323965 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.323879    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:23.323965 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.323904    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:23.323965 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.323964    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret podName:dde8d19a-bea3-48da-9fee-a5353c5e385e nodeName:}" failed. No retries permitted until 2026-07-23 17:45:39.323943178 +0000 UTC m=+34.163451124 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret") pod "global-pull-secret-syncer-tcqbt" (UID: "dde8d19a-bea3-48da-9fee-a5353c5e385e") : object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:23.324160 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.323985    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs podName:59bb77c5-cb7e-41f8-81dc-aef0841f958f nodeName:}" failed. No retries permitted until 2026-07-23 17:45:39.323975004 +0000 UTC m=+34.163482929 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs") pod "network-metrics-daemon-vfv7s" (UID: "59bb77c5-cb7e-41f8-81dc-aef0841f958f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:23.424309 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:23.424275    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:23.424478 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.424412    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 17:45:23.424478 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.424429    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 17:45:23.424478 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.424439    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5vzb7 for pod openshift-network-diagnostics/network-check-target-mzb6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:23.424571 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.424486    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7 podName:e0f08cf4-479e-4017-9776-98f34bd6fef3 nodeName:}" failed. No retries permitted until 2026-07-23 17:45:39.424473492 +0000 UTC m=+34.263981421 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vzb7" (UniqueName: "kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7") pod "network-check-target-mzb6t" (UID: "e0f08cf4-479e-4017-9776-98f34bd6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:23.778624 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:23.778542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:23.778790 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:23.778665    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:24.778257 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:24.778220    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:24.778257 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:24.778247    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:24.778789 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:24.778345    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:24.778789 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:24.778520    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:25.779465 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:25.779428    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:25.779947 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:25.779523    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:26.778667 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:26.778628    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:26.778844 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:26.778680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:26.778844 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:26.778762    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:26.778938 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:26.778875    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:27.778912 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:27.778870    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:27.779328 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:27.778983    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:28.778827 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.778471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:28.779157 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.778499    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:28.779157 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:28.778957    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:28.779157 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:28.779058    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:28.947469 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.947434    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rcf5c" event={"ID":"88012f00-e808-4fe4-9fbe-f0825cfd0ff7","Type":"ContainerStarted","Data":"d335df477ed00685d9cca9d70db3a5748f0704e3305fc5f7c24dd06943863cf6"}
Jul 23 17:45:28.948902 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.948873    2582 generic.go:356] "Generic (PLEG): container finished" podID="ec002160-229b-4bc8-a52f-12354c87c023" containerID="eb1758a7ff74f16e02d8743f995210daf2192d407163f12a91e72bf0ff7affe9" exitCode=0
Jul 23 17:45:28.949034 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.948952    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerDied","Data":"eb1758a7ff74f16e02d8743f995210daf2192d407163f12a91e72bf0ff7affe9"}
Jul 23 17:45:28.950426 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.950360    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-94rjj" event={"ID":"69f296a4-0368-4e06-b923-fa7368469182","Type":"ContainerStarted","Data":"e141c0f3cf90d9ef5ea71be59b54c3084620c6bb9a7b68a735dded9ec21e5baf"}
Jul 23 17:45:28.952005 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.951978    2582 generic.go:356] "Generic (PLEG): container finished" podID="0d1a6f68d7fb21bfef8af77aa3c920bb" containerID="6a58aba325ff91e790ce105255b348f1b70a8068cb9eb535b02008d263d22fc8" exitCode=0
Jul 23 17:45:28.952099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.952060    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal" event={"ID":"0d1a6f68d7fb21bfef8af77aa3c920bb","Type":"ContainerDied","Data":"6a58aba325ff91e790ce105255b348f1b70a8068cb9eb535b02008d263d22fc8"}
Jul 23 17:45:28.953720 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.953685    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9" event={"ID":"a672b3c6-b7ef-4f54-b17b-6586ee8d7868","Type":"ContainerStarted","Data":"0797347c6363197ca5988a0ab4beca5c299a1938a7f12ec3148e6657ca2eaae7"}
Jul 23 17:45:28.955667 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.955643    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-l79t9" event={"ID":"26784fee-febb-4712-9906-14c5a6d9a352","Type":"ContainerStarted","Data":"b1dab5ecf75bc4d51e74d1ddca3c67cdbe8b0878efb4d43313f8a80468c77ef7"}
Jul 23 17:45:28.956902 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.956883    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rb9ft" event={"ID":"24b62c73-baa3-4159-9529-58aa59ebf21d","Type":"ContainerStarted","Data":"a090222d58054e43aece30e424d19263df8098e89e735ef101ee5ef50da9490f"}
Jul 23 17:45:28.959806 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.959782    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"5e204f4b33367d8c7d4ddb694156a517a187643159df3bbb682267780fa00141"}
Jul 23 17:45:28.959876 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.959862    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"eb40e8403485e6d0985e2c5fd96edfb96aec786b111d98df2112ab338d7a3e32"}
Jul 23 17:45:28.959929 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.959912    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"99647987207cb8e9104118ba259967bf3fedc6ec1616d2679438ea10a49dd0a3"}
Jul 23 17:45:28.959977 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.959961    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"756a952cf66c3db312a46ea308d4e71a2d795c8d356f9d4d422f20a5ca82f877"}
Jul 23 17:45:28.960025 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.960010    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"d52030e9597669ff7b0ff165c9069069d9049ab27e9c8aa0faeb84afe71283bf"}
Jul 23 17:45:28.960077 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.960062    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"7597fd9f3d3ca1434629133d81c09666abfbe6273e3e279d3b1f3b219e758f46"}
Jul 23 17:45:28.961020 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.961001    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xn4ct" event={"ID":"ed8d602a-92db-4f5e-86a3-7f25c4209c6e","Type":"ContainerStarted","Data":"9c1800675423758ae8b2b1b3f9d59352bb8239ef87751c167b3142e46b49a7ac"}
Jul 23 17:45:28.962209 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.962191    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal" event={"ID":"845580dd794356674919cca719ca31b8","Type":"ContainerStarted","Data":"4dceeb94e43c8fcf10ef67253f2519241723e8c02a0fc508c2ee3dc3f3081ae1"}
Jul 23 17:45:28.998084 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:28.997991    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rcf5c" podStartSLOduration=2.909949731 podStartE2EDuration="22.997969303s" totalImagesPullingTime="20.088019572s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:28.972331318 +0000 UTC m=+23.811839270" watchObservedRunningTime="2026-07-23 17:45:28.997969303 +0000 UTC m=+23.837477251"
Jul 23 17:45:29.077922 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.077650    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-l79t9" podStartSLOduration=3.067082702 podStartE2EDuration="23.077633588s" totalImagesPullingTime="20.010550886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:29.077199195 +0000 UTC m=+23.916707142" watchObservedRunningTime="2026-07-23 17:45:29.077633588 +0000 UTC m=+23.917141537"
Jul 23 17:45:29.141496 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.141406    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rb9ft" podStartSLOduration=3.14754374 podStartE2EDuration="23.141372302s" totalImagesPullingTime="19.993828562s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:29.105192582 +0000 UTC m=+23.944700527" watchObservedRunningTime="2026-07-23 17:45:29.141372302 +0000 UTC m=+23.980880248"
Jul 23 17:45:29.142861 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.141922    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-94rjj" podStartSLOduration=3.143370005 podStartE2EDuration="23.141906694s" totalImagesPullingTime="19.998536689s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:29.141535041 +0000 UTC m=+23.981042990" watchObservedRunningTime="2026-07-23 17:45:29.141906694 +0000 UTC m=+23.981414643"
Jul 23 17:45:29.196094 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.196040    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xn4ct" podStartSLOduration=3.227307555 podStartE2EDuration="23.196019249s" totalImagesPullingTime="19.968711694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:29.173752424 +0000 UTC m=+24.013260369" watchObservedRunningTime="2026-07-23 17:45:29.196019249 +0000 UTC m=+24.035527198"
Jul 23 17:45:29.778844 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.778739    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:29.779022 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:29.778873    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:29.930566 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.930541    2582 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 17:45:29.966292 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.966257    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal" event={"ID":"0d1a6f68d7fb21bfef8af77aa3c920bb","Type":"ContainerStarted","Data":"b33628ae78702588dcf1d65c92f481a0c325697a897c38cdff4f70819fd4456f"}
Jul 23 17:45:29.967963 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.967937    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9" event={"ID":"a672b3c6-b7ef-4f54-b17b-6586ee8d7868","Type":"ContainerStarted","Data":"3b2ac994ac5c8ca76ea26fa4ebe8d97f78c506089d9dc7d0f176ceb59b8c4cef"}
Jul 23 17:45:29.969336 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.969293    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-nb5kz" event={"ID":"ebfdf19f-561d-4ee7-838f-1135e9e2df8e","Type":"ContainerStarted","Data":"0546fd1cffd1a3d7cfd56aa02cac756f37f8c1e072701d1a6cc255f0792a9aa7"}
Jul 23 17:45:29.992733 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:29.992675    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-151.ec2.internal" podStartSLOduration=22.992658663 podStartE2EDuration="22.992658663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:29.196168223 +0000 UTC m=+24.035676160" watchObservedRunningTime="2026-07-23 17:45:29.992658663 +0000 UTC m=+24.832166624"
Jul 23 17:45:30.018658 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:30.018608    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-151.ec2.internal" podStartSLOduration=23.018591601 podStartE2EDuration="23.018591601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:29.992989828 +0000 UTC m=+24.832497776" watchObservedRunningTime="2026-07-23 17:45:30.018591601 +0000 UTC m=+24.858099545"
Jul 23 17:45:30.640677 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:30.640588    2582 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T17:45:29.930559831Z","UUID":"1e481448-a190-4687-8512-aec2c26b66f3","Handler":null,"Name":"","Endpoint":""}
Jul 23 17:45:30.641989 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:30.641974    2582 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 17:45:30.642071 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:30.642009    2582 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 17:45:30.778939 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:30.778896    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:30.778939 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:30.778920    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:30.779171 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:30.779040    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:30.779171 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:30.779153    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:31.778315 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:31.778279    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:31.778768 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:31.778411    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:31.975413 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:31.975358    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9" event={"ID":"a672b3c6-b7ef-4f54-b17b-6586ee8d7868","Type":"ContainerStarted","Data":"7ec7da296def908fa80b32cf7f28de44f802afd3b895c17c55a15a71742acb8e"}
Jul 23 17:45:31.978377 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:31.978350    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"527f479f623301bd71f9baa4a8a09d1706c0532fa568894984c9725eae4419f9"}
Jul 23 17:45:31.997241 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:31.997193    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-nb5kz" podStartSLOduration=6.032435171 podStartE2EDuration="25.99717757s" totalImagesPullingTime="19.964742399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:30.019849537 +0000 UTC m=+24.859357497" watchObservedRunningTime="2026-07-23 17:45:31.99717757 +0000 UTC m=+26.836685518"
Jul 23 17:45:32.341340 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:32.341301    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:32.341869 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:32.341850    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:32.359590 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:32.359538    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8lcv9" podStartSLOduration=3.856564464 podStartE2EDuration="26.359520287s" totalImagesPullingTime="22.502955823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:31.997456879 +0000 UTC m=+26.836964828" watchObservedRunningTime="2026-07-23 17:45:32.359520287 +0000 UTC m=+27.199028242"
Jul 23 17:45:32.779021 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:32.778929    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:32.779521 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:32.778926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:32.779521 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:32.779058    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:32.779521 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:32.779166    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:32.981444 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:32.981401    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:32.981887 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:32.981868    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xn4ct"
Jul 23 17:45:33.778753 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:33.778462    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:33.778916 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:33.778797    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:33.987561 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:33.987521    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" event={"ID":"aa5549a4-15ae-44dd-9c68-e46430112f3c","Type":"ContainerStarted","Data":"d8f31162dad0f10f86f561362fa0f68d4ca1c6d60c0765ca04cb24152d439dcd"}
Jul 23 17:45:34.112944 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:34.112842    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" podStartSLOduration=8.085947211 podStartE2EDuration="28.112819622s" totalImagesPullingTime="20.026872411s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:34.110186478 +0000 UTC m=+28.949694440" watchObservedRunningTime="2026-07-23 17:45:34.112819622 +0000 UTC m=+28.952327568"
Jul 23 17:45:34.778720 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:34.778680    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:34.778910 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:34.778746    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:34.778910 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:34.778835    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:34.779020 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:34.778965    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:34.989794 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:34.989765    2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 23 17:45:34.990365 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:34.990301    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:34.990409 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:34.990387    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:35.006938 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:35.006910    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:35.007088 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:35.007032    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:35.779525 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:35.779486    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:35.779701 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:35.779602    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:35.991633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:35.991603    2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 23 17:45:36.778390 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:36.778364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:36.778495 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:36.778364    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:36.778495 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:36.778469    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:36.778594 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:36.778567    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:36.998582 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:36.998506    2582 generic.go:356] "Generic (PLEG): container finished" podID="ec002160-229b-4bc8-a52f-12354c87c023" containerID="e250a5f61f7429831bf0f82a3707fa99a3479d13b550532546add417167ab7e3" exitCode=0
Jul 23 17:45:36.998582 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:36.998565    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerDied","Data":"e250a5f61f7429831bf0f82a3707fa99a3479d13b550532546add417167ab7e3"}
Jul 23 17:45:36.999097 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:36.998700    2582 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 23 17:45:37.129034 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:37.129000    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:45:37.778389 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:37.778347    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:37.778575 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:37.778478    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:38.010346 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:38.010306    2582 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8" podUID="aa5549a4-15ae-44dd-9c68-e46430112f3c" containerName="ovnkube-controller" probeResult="failure" output=""
Jul 23 17:45:38.778503 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:38.778464    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:38.778700 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:38.778463    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:38.778700 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:38.778627    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:38.778833 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:38.778757    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:39.004217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:39.004186    2582 generic.go:356] "Generic (PLEG): container finished" podID="ec002160-229b-4bc8-a52f-12354c87c023" containerID="14093cf2a4c10379a99c06ebfe587bc55c85e234e40f71433ee57bf1c4a36929" exitCode=0
Jul 23 17:45:39.004217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:39.004230    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerDied","Data":"14093cf2a4c10379a99c06ebfe587bc55c85e234e40f71433ee57bf1c4a36929"}
Jul 23 17:45:39.346499 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:39.346464    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:39.346880 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:39.346519    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:39.346880 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.346601    2582 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:39.346880 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.346664    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret podName:dde8d19a-bea3-48da-9fee-a5353c5e385e nodeName:}" failed. No retries permitted until 2026-07-23 17:46:11.346649874 +0000 UTC m=+66.186157801 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret") pod "global-pull-secret-syncer-tcqbt" (UID: "dde8d19a-bea3-48da-9fee-a5353c5e385e") : object "kube-system"/"original-pull-secret" not registered
Jul 23 17:45:39.346880 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.346609    2582 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:39.346880 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.346744    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs podName:59bb77c5-cb7e-41f8-81dc-aef0841f958f nodeName:}" failed. No retries permitted until 2026-07-23 17:46:11.346732931 +0000 UTC m=+66.186240857 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs") pod "network-metrics-daemon-vfv7s" (UID: "59bb77c5-cb7e-41f8-81dc-aef0841f958f") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 17:45:39.447499 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:39.447454    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:39.447654 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.447590    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 17:45:39.447654 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.447606    2582 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 17:45:39.447654 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.447616    2582 projected.go:196] Error preparing data for projected volume kube-api-access-5vzb7 for pod openshift-network-diagnostics/network-check-target-mzb6t: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:39.447794 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.447665    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7 podName:e0f08cf4-479e-4017-9776-98f34bd6fef3 nodeName:}" failed. No retries permitted until 2026-07-23 17:46:11.44764976 +0000 UTC m=+66.287157685 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5vzb7" (UniqueName: "kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7") pod "network-check-target-mzb6t" (UID: "e0f08cf4-479e-4017-9776-98f34bd6fef3") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 17:45:39.778133 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:39.778048    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:39.778288 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:39.778184    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:40.008908 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:40.008875    2582 generic.go:356] "Generic (PLEG): container finished" podID="ec002160-229b-4bc8-a52f-12354c87c023" containerID="ed0c7adb7312e0e3b4152409033b44abc196451f9c2c1bc61e86c4f51e65dd64" exitCode=0
Jul 23 17:45:40.009040 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:40.008939    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerDied","Data":"ed0c7adb7312e0e3b4152409033b44abc196451f9c2c1bc61e86c4f51e65dd64"}
Jul 23 17:45:40.778485 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:40.778446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:40.778977 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:40.778446    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:40.778977 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:40.778596    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:40.778977 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:40.778727    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:41.778503 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:41.778470    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:41.778962 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:41.778586    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:42.778693 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:42.778654    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:42.779155 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:42.778699    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:42.779155 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:42.778809    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:42.779155 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:42.778918    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:43.779077 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:43.779040    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:43.779537 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:43.779172    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:44.778435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:44.778396    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:44.778435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:44.778424    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:44.778688 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:44.778530    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:44.778768 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:44.778681    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:45.780204 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:45.780167    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:45.780599 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:45.780287    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:46.026130 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:46.026100    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerStarted","Data":"30a634d2171f177c1695c787ea70d01af6e9ee5f378efeb3fac4ecb1748bff8f"}
Jul 23 17:45:46.778013 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:46.777971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:46.778215 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:46.778095    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:46.778215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:46.778145    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:46.778330 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:46.778244    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:47.031259 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:47.031172    2582 generic.go:356] "Generic (PLEG): container finished" podID="ec002160-229b-4bc8-a52f-12354c87c023" containerID="30a634d2171f177c1695c787ea70d01af6e9ee5f378efeb3fac4ecb1748bff8f" exitCode=0
Jul 23 17:45:47.031259 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:47.031222    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerDied","Data":"30a634d2171f177c1695c787ea70d01af6e9ee5f378efeb3fac4ecb1748bff8f"}
Jul 23 17:45:47.778995 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:47.778810    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:47.779184 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:47.779081    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:48.036542 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:48.036461    2582 generic.go:356] "Generic (PLEG): container finished" podID="ec002160-229b-4bc8-a52f-12354c87c023" containerID="aca1bb5346357a5a867c9b836a5356ae1e2dfc83abb4e60b2f23c703592806ac" exitCode=0
Jul 23 17:45:48.036542 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:48.036519    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerDied","Data":"aca1bb5346357a5a867c9b836a5356ae1e2dfc83abb4e60b2f23c703592806ac"}
Jul 23 17:45:48.778537 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:48.778503    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:48.778537 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:48.778533    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:48.778782 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:48.778611    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:48.778881 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:48.778855    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:49.042373 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:49.042287    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-76gc6" event={"ID":"ec002160-229b-4bc8-a52f-12354c87c023","Type":"ContainerStarted","Data":"05cc7ba0a19413f4f2aae898cb42977747c588f216bdda3607d6e00624a8fedb"}
Jul 23 17:45:49.074540 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:49.074490    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-76gc6" podStartSLOduration=6.925349883 podStartE2EDuration="43.074475204s" totalImagesPullingTime="36.149125321s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:45:49.074123425 +0000 UTC m=+43.913631383" watchObservedRunningTime="2026-07-23 17:45:49.074475204 +0000 UTC m=+43.913983149"
Jul 23 17:45:49.778125 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:49.778085    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:49.778293 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:49.778194    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:50.778479 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:50.778441    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:50.778479 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:50.778471    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:50.778915 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:50.778557    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:50.778915 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:50.778625    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:51.778401 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:51.778363    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:51.778573 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:51.778496    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:52.778313 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:52.778273    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:52.778313 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:52.778311    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:52.778524 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:52.778390    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:52.778524 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:52.778486    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:53.385219 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:53.385176    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tcqbt"]
Jul 23 17:45:53.385563 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:53.385353    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:53.385563 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:53.385463    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:53.386029 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:53.385997    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mzb6t"]
Jul 23 17:45:53.386156 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:53.386096    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:53.386212 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:53.386185    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:53.386798 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:53.386771    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vfv7s"]
Jul 23 17:45:53.386890 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:53.386869    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:53.386993 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:53.386967    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:54.778568 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:54.778534    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:54.778568 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:54.778571    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:54.779014 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:54.778642    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:54.779014 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:54.778790    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:55.779496 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:55.779461    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:55.779894 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:55.779558    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:56.778451 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:56.778264    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:56.778633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:56.778271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:56.778633 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:56.778523    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tcqbt" podUID="dde8d19a-bea3-48da-9fee-a5353c5e385e"
Jul 23 17:45:56.778633 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:56.778593    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-mzb6t" podUID="e0f08cf4-479e-4017-9776-98f34bd6fef3"
Jul 23 17:45:57.778036 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:57.777957    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:57.778414 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:45:57.778065    2582 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vfv7s" podUID="59bb77c5-cb7e-41f8-81dc-aef0841f958f"
Jul 23 17:45:57.924680 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:57.924654    2582 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-151.ec2.internal" event="NodeReady"
Jul 23 17:45:57.924855 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:57.924799    2582 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 17:45:57.981064 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:57.981028    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qrgxp"]
Jul 23 17:45:58.005951 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.005915    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-xtzsf"]
Jul 23 17:45:58.006105 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.006043    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.008884 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.008841    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 17:45:58.009022 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.008903    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bjl7v\""
Jul 23 17:45:58.009022 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.008957    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 17:45:58.025972 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.025941    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qrgxp"]
Jul 23 17:45:58.026109 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.025982    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b7hbt"]
Jul 23 17:45:58.026109 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.026052    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.029608 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.029546    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 17:45:58.029608 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.029586    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 17:45:58.029780 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.029640    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-2q846\""
Jul 23 17:45:58.030387 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.030373    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 17:45:58.030771 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.030758    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 17:45:58.050158 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.050130    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xtzsf"]
Jul 23 17:45:58.050291 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.050174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b7hbt"]
Jul 23 17:45:58.050291 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.050250    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.052853 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.052833    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 17:45:58.052966 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.052871    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xf6nc\""
Jul 23 17:45:58.052966 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.052874    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 17:45:58.053275 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.053258    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 17:45:58.079725 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079676    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptbm2\" (UniqueName: \"kubernetes.io/projected/960e922b-95fb-4745-9093-b48be740ce28-kube-api-access-ptbm2\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.079725 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079725    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq8m7\" (UniqueName: \"kubernetes.io/projected/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-kube-api-access-cq8m7\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.079910 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079756    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b-cert\") pod \"ingress-canary-b7hbt\" (UID: \"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b\") " pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.079910 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079771    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/960e922b-95fb-4745-9093-b48be740ce28-config-volume\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.079910 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079788    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.079910 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079801    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/960e922b-95fb-4745-9093-b48be740ce28-tmp-dir\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.079910 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.079910 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079880    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cf97c\" (UniqueName: \"kubernetes.io/projected/5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b-kube-api-access-cf97c\") pod \"ingress-canary-b7hbt\" (UID: \"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b\") " pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.080116 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079936    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-data-volume\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.080116 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079962    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-crio-socket\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.080116 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.079977    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/960e922b-95fb-4745-9093-b48be740ce28-metrics-tls\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.181080 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181043    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptbm2\" (UniqueName: \"kubernetes.io/projected/960e922b-95fb-4745-9093-b48be740ce28-kube-api-access-ptbm2\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.181080 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181083    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cq8m7\" (UniqueName: \"kubernetes.io/projected/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-kube-api-access-cq8m7\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.181367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181111    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b-cert\") pod \"ingress-canary-b7hbt\" (UID: \"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b\") " pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.181367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181127    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/960e922b-95fb-4745-9093-b48be740ce28-config-volume\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.181367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181146    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.181367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181260    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/960e922b-95fb-4745-9093-b48be740ce28-tmp-dir\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.181367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181291    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.181367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cf97c\" (UniqueName: \"kubernetes.io/projected/5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b-kube-api-access-cf97c\") pod \"ingress-canary-b7hbt\" (UID: \"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b\") " pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.181367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181362    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-data-volume\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.181685 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-crio-socket\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.181685 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181419    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/960e922b-95fb-4745-9093-b48be740ce28-metrics-tls\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.181685 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181566    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/960e922b-95fb-4745-9093-b48be740ce28-tmp-dir\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.181685 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181678    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-crio-socket\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.181914 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181779    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/960e922b-95fb-4745-9093-b48be740ce28-config-volume\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.181914 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181824    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.181914 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.181857    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-data-volume\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.185197 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.185175    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/960e922b-95fb-4745-9093-b48be740ce28-metrics-tls\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.185295 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.185208    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.185295 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.185273    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b-cert\") pod \"ingress-canary-b7hbt\" (UID: \"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b\") " pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.189347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.189326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cf97c\" (UniqueName: \"kubernetes.io/projected/5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b-kube-api-access-cf97c\") pod \"ingress-canary-b7hbt\" (UID: \"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b\") " pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.189448 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.189429    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptbm2\" (UniqueName: \"kubernetes.io/projected/960e922b-95fb-4745-9093-b48be740ce28-kube-api-access-ptbm2\") pod \"dns-default-qrgxp\" (UID: \"960e922b-95fb-4745-9093-b48be740ce28\") " pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.189556 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.189541    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq8m7\" (UniqueName: \"kubernetes.io/projected/9a34bd01-1bac-4775-9c86-c2e589b4c3b7-kube-api-access-cq8m7\") pod \"insights-runtime-extractor-xtzsf\" (UID: \"9a34bd01-1bac-4775-9c86-c2e589b4c3b7\") " pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.315834 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.315807    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:45:58.334990 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.334956    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-xtzsf"
Jul 23 17:45:58.358053 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.358023    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b7hbt"
Jul 23 17:45:58.558251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.558221    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b7hbt"]
Jul 23 17:45:58.558875 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.558851    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qrgxp"]
Jul 23 17:45:58.560535 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.560509    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-xtzsf"]
Jul 23 17:45:58.562659 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:58.562617    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d7340a2_0a7a_4786_ac9c_5068e0dc7d8b.slice/crio-9fb520b9fb76bd5bbdef2c1bbc481eca31adb2d270d92e18417d3e1c8ff37d5c WatchSource:0}: Error finding container 9fb520b9fb76bd5bbdef2c1bbc481eca31adb2d270d92e18417d3e1c8ff37d5c: Status 404 returned error can't find the container with id 9fb520b9fb76bd5bbdef2c1bbc481eca31adb2d270d92e18417d3e1c8ff37d5c
Jul 23 17:45:58.562877 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:58.562861    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod960e922b_95fb_4745_9093_b48be740ce28.slice/crio-bee77e0b9957d26b636842b1765758e90755815fc7aa3d2453378db3bb067748 WatchSource:0}: Error finding container bee77e0b9957d26b636842b1765758e90755815fc7aa3d2453378db3bb067748: Status 404 returned error can't find the container with id bee77e0b9957d26b636842b1765758e90755815fc7aa3d2453378db3bb067748
Jul 23 17:45:58.563862 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:45:58.563842    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a34bd01_1bac_4775_9c86_c2e589b4c3b7.slice/crio-1d286b466e2f26a46f03f2f02005f491241a5cabfaa02ec0006b5ee01f2036fd WatchSource:0}: Error finding container 1d286b466e2f26a46f03f2f02005f491241a5cabfaa02ec0006b5ee01f2036fd: Status 404 returned error can't find the container with id 1d286b466e2f26a46f03f2f02005f491241a5cabfaa02ec0006b5ee01f2036fd
Jul 23 17:45:58.777974 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.777939    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:45:58.778159 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.778137    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:45:58.780934 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.780909    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 17:45:58.781203 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.781185    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 17:45:58.781292 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.781185    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mcdkw\""
Jul 23 17:45:58.781292 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:58.781261    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 17:45:59.066104 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:59.066032    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b7hbt" event={"ID":"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b","Type":"ContainerStarted","Data":"9fb520b9fb76bd5bbdef2c1bbc481eca31adb2d270d92e18417d3e1c8ff37d5c"}
Jul 23 17:45:59.067433 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:59.067339    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtzsf" event={"ID":"9a34bd01-1bac-4775-9c86-c2e589b4c3b7","Type":"ContainerStarted","Data":"1d286b466e2f26a46f03f2f02005f491241a5cabfaa02ec0006b5ee01f2036fd"}
Jul 23 17:45:59.068825 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:59.068796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qrgxp" event={"ID":"960e922b-95fb-4745-9093-b48be740ce28","Type":"ContainerStarted","Data":"bee77e0b9957d26b636842b1765758e90755815fc7aa3d2453378db3bb067748"}
Jul 23 17:45:59.784014 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:59.783982    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:45:59.788020 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:59.787985    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 17:45:59.788245 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:45:59.788229    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2xnzn\""
Jul 23 17:46:02.058191 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.057971    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f58894748-nf8v8"]
Jul 23 17:46:02.069849 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.069820    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.072557 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.072517    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 17:46:02.072725 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.072590    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 17:46:02.072725 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.072607    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f58894748-nf8v8"]
Jul 23 17:46:02.072725 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.072590    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-78r8z\""
Jul 23 17:46:02.073979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.073724    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 17:46:02.073979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.073783    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 17:46:02.073979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.073730    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 17:46:02.073979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.073858    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 17:46:02.073979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.073927    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 17:46:02.077976 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.077773    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 17:46:02.114421 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.114380    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-config\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.114588 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.114494    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-oauth-serving-cert\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.114588 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.114525    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-oauth-config\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.114588 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.114562    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9928\" (UniqueName: \"kubernetes.io/projected/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-kube-api-access-q9928\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.114752 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.114590    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-trusted-ca-bundle\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.114752 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.114613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-serving-cert\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.114752 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.114690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-service-ca\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.215920 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.215882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-trusted-ca-bundle\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.215920 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.215926    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-serving-cert\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.215953    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-service-ca\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.215994    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-config\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.216062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-oauth-serving-cert\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.216089    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-oauth-config\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.216116    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9928\" (UniqueName: \"kubernetes.io/projected/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-kube-api-access-q9928\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.216829    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-service-ca\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.216862    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-config\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.216881 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.216875    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-oauth-serving-cert\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.217276 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.217253    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-trusted-ca-bundle\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.219350 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.219319    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-oauth-config\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.219530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.219507    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-serving-cert\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.226522 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.226499    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9928\" (UniqueName: \"kubernetes.io/projected/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-kube-api-access-q9928\") pod \"console-5f58894748-nf8v8\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") " pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:02.381853 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:02.381768    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:04.244553 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.244522    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f58894748-nf8v8"]
Jul 23 17:46:04.769323 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.768797    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"]
Jul 23 17:46:04.771493 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.771065    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.774443 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.774305    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kz8dg\""
Jul 23 17:46:04.775662 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.775640    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 17:46:04.776483 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.775893    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 23 17:46:04.776483 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.776112    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 17:46:04.776483 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.776307    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 23 17:46:04.776483 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.776456    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 17:46:04.782370 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.782347    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"]
Jul 23 17:46:04.837509 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.837471    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnhf4\" (UniqueName: \"kubernetes.io/projected/86ff8c6e-49f8-4662-8e95-56cabf77cde1-kube-api-access-gnhf4\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.837742 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.837609    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.837742 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.837645    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.837742 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.837673    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86ff8c6e-49f8-4662-8e95-56cabf77cde1-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.939023 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.938980    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnhf4\" (UniqueName: \"kubernetes.io/projected/86ff8c6e-49f8-4662-8e95-56cabf77cde1-kube-api-access-gnhf4\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.939217 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.939173    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.939278 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.939214    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.939278 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.939243    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86ff8c6e-49f8-4662-8e95-56cabf77cde1-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.939489 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:46:04.939449    2582 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Jul 23 17:46:04.939587 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:46:04.939521    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-tls podName:86ff8c6e-49f8-4662-8e95-56cabf77cde1 nodeName:}" failed. No retries permitted until 2026-07-23 17:46:05.439501812 +0000 UTC m=+60.279009737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-5gnmj" (UID: "86ff8c6e-49f8-4662-8e95-56cabf77cde1") : secret "prometheus-operator-tls" not found
Jul 23 17:46:04.940211 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.940184    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86ff8c6e-49f8-4662-8e95-56cabf77cde1-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.942651 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.942603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:04.949250 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:04.949222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnhf4\" (UniqueName: \"kubernetes.io/projected/86ff8c6e-49f8-4662-8e95-56cabf77cde1-kube-api-access-gnhf4\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:05.088647 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.088604    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtzsf" event={"ID":"9a34bd01-1bac-4775-9c86-c2e589b4c3b7","Type":"ContainerStarted","Data":"b684e6ab67d358279429bf848d1bcc24f65909e094ac363e99a75e2088d28a06"}
Jul 23 17:46:05.090552 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.090518    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qrgxp" event={"ID":"960e922b-95fb-4745-9093-b48be740ce28","Type":"ContainerStarted","Data":"39081ad63c76826c17be51c0abfd92efe8220265169c70b3b6c568724abac71d"}
Jul 23 17:46:05.090692 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.090576    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qrgxp" event={"ID":"960e922b-95fb-4745-9093-b48be740ce28","Type":"ContainerStarted","Data":"5b1da20d775b31d784a700ac1c6236e3fb58dc0faf3832d81e3b72cd620c2ca8"}
Jul 23 17:46:05.090891 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.090874    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:46:05.092942 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.092915    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b7hbt" event={"ID":"5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b","Type":"ContainerStarted","Data":"40aa324ec45bb411168f7bc8b707393c33cde65754a16510e5f408316f69a804"}
Jul 23 17:46:05.094294 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.094268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58894748-nf8v8" event={"ID":"d5c6606c-3e6d-4b9a-8f0e-af41e757c992","Type":"ContainerStarted","Data":"63cf3b953ebb6eff36a156edf2c5c1dcbf0c9d7efc354aa2021d6e5e5bfb81b0"}
Jul 23 17:46:05.114949 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.114885    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qrgxp" podStartSLOduration=2.909183047 podStartE2EDuration="8.11486245s" totalImagesPullingTime="5.205679403s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:05.114212439 +0000 UTC m=+59.953720388" watchObservedRunningTime="2026-07-23 17:46:05.11486245 +0000 UTC m=+59.954370399"
Jul 23 17:46:05.136472 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.136406    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b7hbt" podStartSLOduration=2.607337457 podStartE2EDuration="8.136381096s" totalImagesPullingTime="5.529043639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:45:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:05.136010718 +0000 UTC m=+59.975518669" watchObservedRunningTime="2026-07-23 17:46:05.136381096 +0000 UTC m=+59.975889027"
Jul 23 17:46:05.445015 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.444928    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:05.447633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.447607    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/86ff8c6e-49f8-4662-8e95-56cabf77cde1-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-5gnmj\" (UID: \"86ff8c6e-49f8-4662-8e95-56cabf77cde1\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:05.688458 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.688423    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-kz8dg\""
Jul 23 17:46:05.696023 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.695963    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"
Jul 23 17:46:05.851536 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:05.851500    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj"]
Jul 23 17:46:07.221063 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:07.221020    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86ff8c6e_49f8_4662_8e95_56cabf77cde1.slice/crio-14ce7f016cbc7596f2d4dae16a4695f51a8f2024903a1aeef50e4e2afe6cad47 WatchSource:0}: Error finding container 14ce7f016cbc7596f2d4dae16a4695f51a8f2024903a1aeef50e4e2afe6cad47: Status 404 returned error can't find the container with id 14ce7f016cbc7596f2d4dae16a4695f51a8f2024903a1aeef50e4e2afe6cad47
Jul 23 17:46:08.013432 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:08.013377    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-pnrc8"
Jul 23 17:46:08.104935 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:08.104887    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj" event={"ID":"86ff8c6e-49f8-4662-8e95-56cabf77cde1","Type":"ContainerStarted","Data":"14ce7f016cbc7596f2d4dae16a4695f51a8f2024903a1aeef50e4e2afe6cad47"}
Jul 23 17:46:10.114396 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:10.114356    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtzsf" event={"ID":"9a34bd01-1bac-4775-9c86-c2e589b4c3b7","Type":"ContainerStarted","Data":"5b056abdc09b3f54417a672746b027479ec1aa750f0c4a4c83267a4d803b2347"}
Jul 23 17:46:10.116140 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:10.116111    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58894748-nf8v8" event={"ID":"d5c6606c-3e6d-4b9a-8f0e-af41e757c992","Type":"ContainerStarted","Data":"5266c1c1c5c89e7e9bbcfbc69d5f0e179277227f45933f511e0a599fd69ac6c3"}
Jul 23 17:46:10.117971 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:10.117946    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj" event={"ID":"86ff8c6e-49f8-4662-8e95-56cabf77cde1","Type":"ContainerStarted","Data":"ba44879e35a5895f10b13e7c24715810e2b22e87a66d7189b766a1020e582124"}
Jul 23 17:46:10.118097 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:10.117997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj" event={"ID":"86ff8c6e-49f8-4662-8e95-56cabf77cde1","Type":"ContainerStarted","Data":"38921fca985f333f905d0af0ed761b7795ceeda543fa2414b9cdca8069566cff"}
Jul 23 17:46:10.136402 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:10.136333    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f58894748-nf8v8" podStartSLOduration=3.092849842 podStartE2EDuration="8.136312597s" totalImagesPullingTime="5.043462755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:46:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:10.134567021 +0000 UTC m=+64.974074994" watchObservedRunningTime="2026-07-23 17:46:10.136312597 +0000 UTC m=+64.975820548"
Jul 23 17:46:10.152120 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:10.152066    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-5gnmj" podStartSLOduration=4.079891935 podStartE2EDuration="6.152045979s" totalImagesPullingTime="2.072154044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:46:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:10.150799138 +0000 UTC m=+64.990307119" watchObservedRunningTime="2026-07-23 17:46:10.152045979 +0000 UTC m=+64.991553947"
Jul 23 17:46:11.394886 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.394843    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:46:11.395316 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.394913    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:46:11.398771 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.398748    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 17:46:11.399688 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.399665    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 17:46:11.408232 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.408199    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/dde8d19a-bea3-48da-9fee-a5353c5e385e-original-pull-secret\") pod \"global-pull-secret-syncer-tcqbt\" (UID: \"dde8d19a-bea3-48da-9fee-a5353c5e385e\") " pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:46:11.408379 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.408296    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59bb77c5-cb7e-41f8-81dc-aef0841f958f-metrics-certs\") pod \"network-metrics-daemon-vfv7s\" (UID: \"59bb77c5-cb7e-41f8-81dc-aef0841f958f\") " pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:46:11.495799 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.495757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:46:11.499163 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.499136    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 17:46:11.500087 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.500062    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2xnzn\""
Jul 23 17:46:11.508126 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.508089    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vfv7s"
Jul 23 17:46:11.508301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.508260    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 17:46:11.519084 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.519047    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vzb7\" (UniqueName: \"kubernetes.io/projected/e0f08cf4-479e-4017-9776-98f34bd6fef3-kube-api-access-5vzb7\") pod \"network-check-target-mzb6t\" (UID: \"e0f08cf4-479e-4017-9776-98f34bd6fef3\") " pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:46:11.692215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.692141    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-mcdkw\""
Jul 23 17:46:11.693187 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.693164    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tcqbt"
Jul 23 17:46:11.699931 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:11.699907    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:46:12.228666 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.228569    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-sllgk"]
Jul 23 17:46:12.236300 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.236271    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.240212 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.239915    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 23 17:46:12.240883 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.240858    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 23 17:46:12.241003 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.240866    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-sw9ft\""
Jul 23 17:46:12.241003 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.240977    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 23 17:46:12.255309 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.255281    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qrmgs"]
Jul 23 17:46:12.258329 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.258308    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.261124 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.261093    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 17:46:12.261252 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.261156    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xxs4p\""
Jul 23 17:46:12.261252 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.261158    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 17:46:12.261481 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.261458    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 17:46:12.301478 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301441    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-tls\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.301680 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301489    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-sys\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.301680 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301525    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-textfile\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.301680 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301640    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ff0b11-a868-4bb3-a040-ece238d061d7-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.301864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqsx\" (UniqueName: \"kubernetes.io/projected/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-api-access-vqqsx\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.301864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-root\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.301864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301779    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-wtmp\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.301864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301818    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ff0b11-a868-4bb3-a040-ece238d061d7-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.301864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301850    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.302103 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301885    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.302103 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301912    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-metrics-client-ca\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.302103 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301929    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6lcb\" (UniqueName: \"kubernetes.io/projected/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-kube-api-access-t6lcb\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.302103 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.301961    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.302103 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.302000    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.302103 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.302048    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.309094 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.309065    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-sllgk"]
Jul 23 17:46:12.382689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.382629    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:12.382892 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.382731    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:12.384446 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.384414    2582 patch_prober.go:28] interesting pod/console-5f58894748-nf8v8 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 23 17:46:12.384589 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.384473    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5f58894748-nf8v8" podUID="d5c6606c-3e6d-4b9a-8f0e-af41e757c992" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 23 17:46:12.403236 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-tls\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-sys\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-textfile\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:46:12.403354    2582 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403398    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-sys\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:46:12.403431    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-tls podName:211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0 nodeName:}" failed. No retries permitted until 2026-07-23 17:46:12.90340638 +0000 UTC m=+67.742914306 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-tls") pod "node-exporter-qrmgs" (UID: "211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0") : secret "node-exporter-tls" not found
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ff0b11-a868-4bb3-a040-ece238d061d7-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403473    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqsx\" (UniqueName: \"kubernetes.io/projected/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-api-access-vqqsx\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403503    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-root\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403528    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-wtmp\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403566    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ff0b11-a868-4bb3-a040-ece238d061d7-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403600    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.403732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403677    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-metrics-client-ca\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403787    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-wtmp\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403803    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/d0ff0b11-a868-4bb3-a040-ece238d061d7-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403837    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-root\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403944    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-textfile\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.403987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6lcb\" (UniqueName: \"kubernetes.io/projected/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-kube-api-access-t6lcb\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.404020    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.404052    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.404243    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-metrics-client-ca\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.404298    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.404347 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.404340    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d0ff0b11-a868-4bb3-a040-ece238d061d7-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.404775 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.404501    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-accelerators-collector-config\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.405018 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.404956    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.407290 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.407259    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.407402 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.407360    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.407692 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.407665    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.422123 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.422095    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6lcb\" (UniqueName: \"kubernetes.io/projected/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-kube-api-access-t6lcb\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.422542 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.422519    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqsx\" (UniqueName: \"kubernetes.io/projected/d0ff0b11-a868-4bb3-a040-ece238d061d7-kube-api-access-vqqsx\") pod \"kube-state-metrics-56c5594674-sllgk\" (UID: \"d0ff0b11-a868-4bb3-a040-ece238d061d7\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.547922 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.547830    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk"
Jul 23 17:46:12.910763 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.910726    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-tls\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.913177 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.913145    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0-node-exporter-tls\") pod \"node-exporter-qrmgs\" (UID: \"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0\") " pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:12.915504 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:12.915432    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tcqbt"]
Jul 23 17:46:12.918373 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:12.918347    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddde8d19a_bea3_48da_9fee_a5353c5e385e.slice/crio-c2f25597dd826ee4b231808a58614ef53eea1aa0ca9dec2688769759cd9a4db1 WatchSource:0}: Error finding container c2f25597dd826ee4b231808a58614ef53eea1aa0ca9dec2688769759cd9a4db1: Status 404 returned error can't find the container with id c2f25597dd826ee4b231808a58614ef53eea1aa0ca9dec2688769759cd9a4db1
Jul 23 17:46:13.129990 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:13.129954    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-xtzsf" event={"ID":"9a34bd01-1bac-4775-9c86-c2e589b4c3b7","Type":"ContainerStarted","Data":"9a23dd51dbea089f7a0f8b24c7f75da0280f2484b034bfb04a0b0dcefe425a6b"}
Jul 23 17:46:13.131057 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:13.131025    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tcqbt" event={"ID":"dde8d19a-bea3-48da-9fee-a5353c5e385e","Type":"ContainerStarted","Data":"c2f25597dd826ee4b231808a58614ef53eea1aa0ca9dec2688769759cd9a4db1"}
Jul 23 17:46:13.168386 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:13.168352    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qrmgs"
Jul 23 17:46:13.172042 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:13.172018    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-mzb6t"]
Jul 23 17:46:13.174545 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:13.174497    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-xtzsf" podStartSLOduration=2.136642952 podStartE2EDuration="16.17448235s" totalImagesPullingTime="14.037839398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:57 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:13.173172386 +0000 UTC m=+68.012680335" watchObservedRunningTime="2026-07-23 17:46:13.17448235 +0000 UTC m=+68.013990275"
Jul 23 17:46:13.177933 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:13.177906    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod211e2e1a_fa7f_4d51_ab73_e1cf89b94dc0.slice/crio-0c4eca9b7e42cfa7881d3487110fbbd9472f15c65c64b85c986229bfbb9f2673 WatchSource:0}: Error finding container 0c4eca9b7e42cfa7881d3487110fbbd9472f15c65c64b85c986229bfbb9f2673: Status 404 returned error can't find the container with id 0c4eca9b7e42cfa7881d3487110fbbd9472f15c65c64b85c986229bfbb9f2673
Jul 23 17:46:13.182693 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:13.182671    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vfv7s"]
Jul 23 17:46:13.183829 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:13.183810    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-sllgk"]
Jul 23 17:46:13.185519 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:13.185493    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59bb77c5_cb7e_41f8_81dc_aef0841f958f.slice/crio-00b0bac0c459c40713c96fb275900abedc677a42ed37c606642ea9c2bf1d0929 WatchSource:0}: Error finding container 00b0bac0c459c40713c96fb275900abedc677a42ed37c606642ea9c2bf1d0929: Status 404 returned error can't find the container with id 00b0bac0c459c40713c96fb275900abedc677a42ed37c606642ea9c2bf1d0929
Jul 23 17:46:13.186119 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:13.186100    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0ff0b11_a868_4bb3_a040_ece238d061d7.slice/crio-aa6192d6f1f5c7ac1d36c2acf13c529b1257c0eb40bbc5804285d524d659cf9c WatchSource:0}: Error finding container aa6192d6f1f5c7ac1d36c2acf13c529b1257c0eb40bbc5804285d524d659cf9c: Status 404 returned error can't find the container with id aa6192d6f1f5c7ac1d36c2acf13c529b1257c0eb40bbc5804285d524d659cf9c
Jul 23 17:46:13.187078 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:13.187060    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f08cf4_479e_4017_9776_98f34bd6fef3.slice/crio-e78de4b5c340797e6d215233474aca55a427b6fb575be768c4239c2749d4a801 WatchSource:0}: Error finding container e78de4b5c340797e6d215233474aca55a427b6fb575be768c4239c2749d4a801: Status 404 returned error can't find the container with id e78de4b5c340797e6d215233474aca55a427b6fb575be768c4239c2749d4a801
Jul 23 17:46:14.137340 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:14.137300    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrmgs" event={"ID":"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0","Type":"ContainerStarted","Data":"0c4eca9b7e42cfa7881d3487110fbbd9472f15c65c64b85c986229bfbb9f2673"}
Jul 23 17:46:14.139180 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:14.139133    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vfv7s" event={"ID":"59bb77c5-cb7e-41f8-81dc-aef0841f958f","Type":"ContainerStarted","Data":"00b0bac0c459c40713c96fb275900abedc677a42ed37c606642ea9c2bf1d0929"}
Jul 23 17:46:14.140870 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:14.140796    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mzb6t" event={"ID":"e0f08cf4-479e-4017-9776-98f34bd6fef3","Type":"ContainerStarted","Data":"e78de4b5c340797e6d215233474aca55a427b6fb575be768c4239c2749d4a801"}
Jul 23 17:46:14.143280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:14.143253    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk" event={"ID":"d0ff0b11-a868-4bb3-a040-ece238d061d7","Type":"ContainerStarted","Data":"aa6192d6f1f5c7ac1d36c2acf13c529b1257c0eb40bbc5804285d524d659cf9c"}
Jul 23 17:46:15.100159 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:15.099769    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qrgxp"
Jul 23 17:46:16.152828 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.152785    2582 generic.go:356] "Generic (PLEG): container finished" podID="211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0" containerID="0973e41d8ceedf47cd83eb2d56ea597f4ad1c04b1609cb29804102ffcce35c03" exitCode=0
Jul 23 17:46:16.153286 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.152880    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrmgs" event={"ID":"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0","Type":"ContainerDied","Data":"0973e41d8ceedf47cd83eb2d56ea597f4ad1c04b1609cb29804102ffcce35c03"}
Jul 23 17:46:16.155062 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.155034    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vfv7s" event={"ID":"59bb77c5-cb7e-41f8-81dc-aef0841f958f","Type":"ContainerStarted","Data":"eab06587d34cb54ad3b55b97542307b43b543990aeb58e6a53ae0c6d24bb1d55"}
Jul 23 17:46:16.155180 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.155089    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vfv7s" event={"ID":"59bb77c5-cb7e-41f8-81dc-aef0841f958f","Type":"ContainerStarted","Data":"dedbeb36ee644e7f35565bd7f3313e4bacb152322b0981477fb74c53ae5b8228"}
Jul 23 17:46:16.157322 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.157301    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk" event={"ID":"d0ff0b11-a868-4bb3-a040-ece238d061d7","Type":"ContainerStarted","Data":"67767b9d9004bfc98200c78272bbe061ddc98ef8e55cbdf384e5ad9b5122e03b"}
Jul 23 17:46:16.157412 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.157347    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk" event={"ID":"d0ff0b11-a868-4bb3-a040-ece238d061d7","Type":"ContainerStarted","Data":"42ec13a76755e6cd09320023c24081dded9e1e39b5a823bf6f1b7ae07e15d7f2"}
Jul 23 17:46:16.157412 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.157369    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk" event={"ID":"d0ff0b11-a868-4bb3-a040-ece238d061d7","Type":"ContainerStarted","Data":"21e207110bb8997857368f9c888fa725990838bed76618b44354db8db42b5b7b"}
Jul 23 17:46:16.191932 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.191868    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vfv7s" podStartSLOduration=68.048122214 podStartE2EDuration="1m10.191845974s" totalImagesPullingTime="2.14372376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:16.190499272 +0000 UTC m=+71.030007221" watchObservedRunningTime="2026-07-23 17:46:16.191845974 +0000 UTC m=+71.031353923"
Jul 23 17:46:16.210640 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.210572    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-sllgk" podStartSLOduration=2.068381192 podStartE2EDuration="4.210548423s" totalImagesPullingTime="2.142167231s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:16.209693326 +0000 UTC m=+71.049201276" watchObservedRunningTime="2026-07-23 17:46:16.210548423 +0000 UTC m=+71.050056375"
Jul 23 17:46:16.734528 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.734493    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"]
Jul 23 17:46:16.736792 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.736771    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.739849 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.739782    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 23 17:46:16.739849 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.739804    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 23 17:46:16.740626 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.740600    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 23 17:46:16.740955 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.740936    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-2xqml\""
Jul 23 17:46:16.741059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.740990    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 23 17:46:16.741227 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.741206    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-cu40klrpn99n2\""
Jul 23 17:46:16.747356 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.747293    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"]
Jul 23 17:46:16.854800 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.854757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7289462b-47e1-4b5f-9901-b6248a57f4ec-metrics-server-audit-profiles\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.854996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.854811    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7289462b-47e1-4b5f-9901-b6248a57f4ec-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.854996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.854866    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-client-ca-bundle\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.854996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.854894    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-secret-metrics-server-tls\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.854996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.854927    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2m2x\" (UniqueName: \"kubernetes.io/projected/7289462b-47e1-4b5f-9901-b6248a57f4ec-kube-api-access-t2m2x\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.854996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.854989    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-secret-metrics-server-client-certs\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.855212 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.855016    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7289462b-47e1-4b5f-9901-b6248a57f4ec-audit-log\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.934875 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.934838    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"]
Jul 23 17:46:16.937079 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.937051    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:16.939966 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.939943    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 23 17:46:16.940239 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.940224    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bmn4g\""
Jul 23 17:46:16.956435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956084    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"]
Jul 23 17:46:16.956435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956254    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7289462b-47e1-4b5f-9901-b6248a57f4ec-metrics-server-audit-profiles\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.956435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7289462b-47e1-4b5f-9901-b6248a57f4ec-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.956435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956334    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-client-ca-bundle\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.956435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956360    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-secret-metrics-server-tls\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.956435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956394    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2m2x\" (UniqueName: \"kubernetes.io/projected/7289462b-47e1-4b5f-9901-b6248a57f4ec-kube-api-access-t2m2x\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.956435 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956427    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-secret-metrics-server-client-certs\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.956908 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956448    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7289462b-47e1-4b5f-9901-b6248a57f4ec-audit-log\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.956908 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.956830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7289462b-47e1-4b5f-9901-b6248a57f4ec-audit-log\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.958400 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.957369    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7289462b-47e1-4b5f-9901-b6248a57f4ec-metrics-server-audit-profiles\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.958400 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.957393    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7289462b-47e1-4b5f-9901-b6248a57f4ec-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.960863 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.960840    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-secret-metrics-server-client-certs\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.961059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.961033    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-client-ca-bundle\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.961563 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.961542    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7289462b-47e1-4b5f-9901-b6248a57f4ec-secret-metrics-server-tls\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:16.975788 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:16.975761    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2m2x\" (UniqueName: \"kubernetes.io/projected/7289462b-47e1-4b5f-9901-b6248a57f4ec-kube-api-access-t2m2x\") pod \"metrics-server-59bd5df7f6-vkwhg\" (UID: \"7289462b-47e1-4b5f-9901-b6248a57f4ec\") " pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:17.013893 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.013806    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-658757c466-j7jg5"]
Jul 23 17:46:17.017129 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.017103    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.033754 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.033722    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658757c466-j7jg5"]
Jul 23 17:46:17.048768 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.048735    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:17.057807 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.057772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vnzd\" (UniqueName: \"kubernetes.io/projected/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-kube-api-access-9vnzd\") pod \"monitoring-plugin-7b7b7bc8bf-ghkth\" (UID: \"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:17.057950 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.057848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-ghkth\" (UID: \"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:17.158289 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158250    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-ghkth\" (UID: \"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:17.158289 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158300    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bp872\" (UniqueName: \"kubernetes.io/projected/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-kube-api-access-bp872\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158330    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-serving-cert\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158361    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-oauth-config\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158391    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-trusted-ca-bundle\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158426    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-service-ca\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:46:17.158435    2582 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158458    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-config\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:46:17.158511    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-monitoring-plugin-cert podName:8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3 nodeName:}" failed. No retries permitted until 2026-07-23 17:46:17.658485795 +0000 UTC m=+72.497993725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-ghkth" (UID: "8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3") : secret "monitoring-plugin-cert" not found
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158536    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vnzd\" (UniqueName: \"kubernetes.io/projected/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-kube-api-access-9vnzd\") pod \"monitoring-plugin-7b7b7bc8bf-ghkth\" (UID: \"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:17.158864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.158587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-oauth-serving-cert\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.170186 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.170158    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vnzd\" (UniqueName: \"kubernetes.io/projected/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-kube-api-access-9vnzd\") pod \"monitoring-plugin-7b7b7bc8bf-ghkth\" (UID: \"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:17.259456 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.259418    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-oauth-serving-cert\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.260367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.259909    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bp872\" (UniqueName: \"kubernetes.io/projected/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-kube-api-access-bp872\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.260367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.259951    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-serving-cert\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.260367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.260139    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-oauth-config\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.260367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.260203    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-trusted-ca-bundle\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.260367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.260270    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-service-ca\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.260367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.260320    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-config\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.260756 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.260729    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-oauth-serving-cert\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.261008 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.260982    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-config\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.261110 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.261011    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-trusted-ca-bundle\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.261110 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.261022    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-service-ca\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.263041 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.263016    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-serving-cert\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.263249 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.263230    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-oauth-config\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.275007 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.274922    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bp872\" (UniqueName: \"kubernetes.io/projected/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-kube-api-access-bp872\") pod \"console-658757c466-j7jg5\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") " pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.327824 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.327788    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:17.663842 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.663796    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-ghkth\" (UID: \"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:17.666515 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.666488    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-ghkth\" (UID: \"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:17.849143 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:17.849106    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:18.449034 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.448989    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:46:18.462040 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.462010    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.465783 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.465339    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 17:46:18.465783 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.465619    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 17:46:18.466863 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.466840    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 17:46:18.468090 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.468071    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 17:46:18.468189 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.468091    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 17:46:18.468189 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.468103    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 17:46:18.468647 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.468620    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.473040    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.473282    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.473344    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.473592    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c391mm8sv2ilj\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.473893    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.474112    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bb6tx\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.474328    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 17:46:18.475167 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.474572    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 17:46:18.478608 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.478588    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:46:18.494089 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.494060    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 17:46:18.576030 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.575986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576206 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576037    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576206 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576065    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576206 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576111    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576206 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576140    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576206 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576206 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576199    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576502 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-web-config\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576502 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576502 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576314    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqvk7\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-kube-api-access-kqvk7\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576502 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576404    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576502 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576484    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-config\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576507    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576543    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576587    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576613    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576637    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576664    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576690    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.576760 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.576743    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-config-out\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.681906 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681675    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.681906 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681756    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.681906 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681800    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.681906 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681828    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.681906 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681854    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.681906 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681900    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681922    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-web-config\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681950    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.681974    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqvk7\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-kube-api-access-kqvk7\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682001    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682036    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-config\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682062    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682096    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682148    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682175    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682232    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682271    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682841 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682306    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-config-out\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.682841 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.682352    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.683393 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.683163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.691968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.684546    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.691968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.688697    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.691968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.689315    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.691968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.689865    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.691968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.690187    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.691968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.690463    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.691968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.691096    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.694571 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.694532    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.699633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.696601    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.699633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.696990    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-config\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.699633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.697043    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.699633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.697130    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.699633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.697539    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.699633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.697754    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.699633 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.699412    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-config-out\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.703135 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.702635    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-web-config\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.703135 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.703091    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.707261 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.707195    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.711095 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.710988    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqvk7\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-kube-api-access-kqvk7\") pod \"prometheus-k8s-0\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.777818 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.777638    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:18.781107 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.780655    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-658757c466-j7jg5"]
Jul 23 17:46:18.787258 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:18.787223    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8085a58_02dc_40a1_96a0_5f8171fa6aaf.slice/crio-81b131d4d5a7ebd5b78d22d7fe0d0396cc2d542e0cefc02aa3602aba23a426e1 WatchSource:0}: Error finding container 81b131d4d5a7ebd5b78d22d7fe0d0396cc2d542e0cefc02aa3602aba23a426e1: Status 404 returned error can't find the container with id 81b131d4d5a7ebd5b78d22d7fe0d0396cc2d542e0cefc02aa3602aba23a426e1
Jul 23 17:46:18.835938 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.835761    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"]
Jul 23 17:46:18.839556 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:18.839502    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7289462b_47e1_4b5f_9901_b6248a57f4ec.slice/crio-c97588df5d83523d287a4b337d2b0259fabda63e19126592a9b85c94d97927a4 WatchSource:0}: Error finding container c97588df5d83523d287a4b337d2b0259fabda63e19126592a9b85c94d97927a4: Status 404 returned error can't find the container with id c97588df5d83523d287a4b337d2b0259fabda63e19126592a9b85c94d97927a4
Jul 23 17:46:18.841804 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:18.841783    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"]
Jul 23 17:46:18.848222 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:18.848178    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d5d8fe0_d14b_43f3_8e5d_68d815ff9cc3.slice/crio-f1556549409ce299848dc704378f6a19508330d169bf29df440d1ed8fe3cefc7 WatchSource:0}: Error finding container f1556549409ce299848dc704378f6a19508330d169bf29df440d1ed8fe3cefc7: Status 404 returned error can't find the container with id f1556549409ce299848dc704378f6a19508330d169bf29df440d1ed8fe3cefc7
Jul 23 17:46:19.000301 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.000147    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:46:19.005292 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:46:19.005244    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc48c5570_8627_43d4_9a32_581493a2a345.slice/crio-12a102549777fbbf6f52aeb5b15c9969e537663ef6ddb3af91a69791714b129d WatchSource:0}: Error finding container 12a102549777fbbf6f52aeb5b15c9969e537663ef6ddb3af91a69791714b129d: Status 404 returned error can't find the container with id 12a102549777fbbf6f52aeb5b15c9969e537663ef6ddb3af91a69791714b129d
Jul 23 17:46:19.171236 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.171202    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tcqbt" event={"ID":"dde8d19a-bea3-48da-9fee-a5353c5e385e","Type":"ContainerStarted","Data":"9b6e23f1fe0c3bda58e5602c3040aa3f5132616c15f102d5e3fc3ef03ee2fc41"}
Jul 23 17:46:19.172404 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.172375    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth" event={"ID":"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3","Type":"ContainerStarted","Data":"f1556549409ce299848dc704378f6a19508330d169bf29df440d1ed8fe3cefc7"}
Jul 23 17:46:19.173688 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.173666    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-mzb6t" event={"ID":"e0f08cf4-479e-4017-9776-98f34bd6fef3","Type":"ContainerStarted","Data":"f406d9bee34ee2d90dc729cdd5f0492ca906775c6e91af8b98dd94f3c908c0e5"}
Jul 23 17:46:19.173767 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.173742    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:46:19.175848 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.175813    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrmgs" event={"ID":"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0","Type":"ContainerStarted","Data":"1bd3e43b21abdea643ce10f61ae293e339ad3209a27c9f9c51806e6aef3174e8"}
Jul 23 17:46:19.175960 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.175858    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qrmgs" event={"ID":"211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0","Type":"ContainerStarted","Data":"a2e6a29e8eb590102a5447dac62cccad28ffbe30f00925bb0e6ed6eddc2a8b45"}
Jul 23 17:46:19.176935 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.176917    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerStarted","Data":"12a102549777fbbf6f52aeb5b15c9969e537663ef6ddb3af91a69791714b129d"}
Jul 23 17:46:19.178008 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.177989    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg" event={"ID":"7289462b-47e1-4b5f-9901-b6248a57f4ec","Type":"ContainerStarted","Data":"c97588df5d83523d287a4b337d2b0259fabda63e19126592a9b85c94d97927a4"}
Jul 23 17:46:19.179310 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.179292    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658757c466-j7jg5" event={"ID":"b8085a58-02dc-40a1-96a0-5f8171fa6aaf","Type":"ContainerStarted","Data":"23159b7d834b7c6dcbbf0c924a9c533f7aa7f9d782e37c11f51c94a9da71206b"}
Jul 23 17:46:19.179391 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.179322    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658757c466-j7jg5" event={"ID":"b8085a58-02dc-40a1-96a0-5f8171fa6aaf","Type":"ContainerStarted","Data":"81b131d4d5a7ebd5b78d22d7fe0d0396cc2d542e0cefc02aa3602aba23a426e1"}
Jul 23 17:46:19.191916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.191869    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tcqbt" podStartSLOduration=67.561351691 podStartE2EDuration="1m13.191852457s" totalImagesPullingTime="5.630500766s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:19.190897759 +0000 UTC m=+74.030405708" watchObservedRunningTime="2026-07-23 17:46:19.191852457 +0000 UTC m=+74.031360382"
Jul 23 17:46:19.217850 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.217785    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-658757c466-j7jg5" podStartSLOduration=3.217763558 podStartE2EDuration="3.217763558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:46:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:19.215657876 +0000 UTC m=+74.055165823" watchObservedRunningTime="2026-07-23 17:46:19.217763558 +0000 UTC m=+74.057271508"
Jul 23 17:46:19.265201 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.265086    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-mzb6t" podStartSLOduration=67.917270144 podStartE2EDuration="1m13.265068413s" totalImagesPullingTime="5.347798269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:19.263414116 +0000 UTC m=+74.102922065" watchObservedRunningTime="2026-07-23 17:46:19.265068413 +0000 UTC m=+74.104576395"
Jul 23 17:46:19.297295 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:19.297229    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qrmgs" podStartSLOduration=5.013750267 podStartE2EDuration="7.297209734s" totalImagesPullingTime="2.14917948s" totalInitContainerRuntime="134.279987ms" isStatefulPod=true podCreationTimestamp="2026-07-23 17:46:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:19.295633038 +0000 UTC m=+74.135140986" watchObservedRunningTime="2026-07-23 17:46:19.297209734 +0000 UTC m=+74.136717685"
Jul 23 17:46:21.189487 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.189378    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth" event={"ID":"8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3","Type":"ContainerStarted","Data":"817b1640d8f42c67b4bc8e276b00eea607110e5356ef3d0e531cc55ab7c5409e"}
Jul 23 17:46:21.189948 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.189526    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:21.191122 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.191096    2582 generic.go:356] "Generic (PLEG): container finished" podID="c48c5570-8627-43d4-9a32-581493a2a345" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a" exitCode=0
Jul 23 17:46:21.191234 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.191165    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"}
Jul 23 17:46:21.193015 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.192955    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg" event={"ID":"7289462b-47e1-4b5f-9901-b6248a57f4ec","Type":"ContainerStarted","Data":"050cf0bfe185260c851db94c0a2c5aee13fde1c4f352b4e663adcfdbeed3842b"}
Jul 23 17:46:21.195973 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.195953    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth"
Jul 23 17:46:21.211178 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.211132    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-ghkth" podStartSLOduration=3.161927487 podStartE2EDuration="5.211117313s" totalImagesPullingTime="2.049189826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:21.209955849 +0000 UTC m=+76.049463798" watchObservedRunningTime="2026-07-23 17:46:21.211117313 +0000 UTC m=+76.050625258"
Jul 23 17:46:21.228965 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.228909    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg" podStartSLOduration=3.172778153 podStartE2EDuration="5.228896604s" totalImagesPullingTime="2.056118451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:46:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:21.228822319 +0000 UTC m=+76.068330268" watchObservedRunningTime="2026-07-23 17:46:21.228896604 +0000 UTC m=+76.068404552"
Jul 23 17:46:21.563408 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:21.563370    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f58894748-nf8v8"]
Jul 23 17:46:25.214887 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:25.214845    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerStarted","Data":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"}
Jul 23 17:46:25.215329 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:25.214974    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerStarted","Data":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"}
Jul 23 17:46:27.226340 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.226223    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerStarted","Data":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"}
Jul 23 17:46:27.226340 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.226289    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerStarted","Data":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"}
Jul 23 17:46:27.226340 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.226313    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerStarted","Data":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"}
Jul 23 17:46:27.226887 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.226376    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerStarted","Data":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"}
Jul 23 17:46:27.280801 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.280735    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.308905468 podStartE2EDuration="9.280695773s" totalImagesPullingTime="6.951694687s" totalInitContainerRuntime="20.095618ms" isStatefulPod=false podCreationTimestamp="2026-07-23 17:46:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:46:27.275546006 +0000 UTC m=+82.115053968" watchObservedRunningTime="2026-07-23 17:46:27.280695773 +0000 UTC m=+82.120203716"
Jul 23 17:46:27.328894 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.328855    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:27.329057 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.328935    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:27.333945 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:27.333924    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:28.234411 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:28.234382    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:28.778039 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:28.777996    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:46:32.135401 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:32.135365    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658757c466-j7jg5"]
Jul 23 17:46:37.049821 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:37.049772    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:37.050225 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:37.049848    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:46.586638 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:46.586589    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5f58894748-nf8v8" podUID="d5c6606c-3e6d-4b9a-8f0e-af41e757c992" containerName="console" containerID="cri-o://5266c1c1c5c89e7e9bbcfbc69d5f0e179277227f45933f511e0a599fd69ac6c3" gracePeriod=15
Jul 23 17:46:47.300642 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.300610    2582 generic.go:356] "Generic (PLEG): container finished" podID="d5c6606c-3e6d-4b9a-8f0e-af41e757c992" containerID="5266c1c1c5c89e7e9bbcfbc69d5f0e179277227f45933f511e0a599fd69ac6c3" exitCode=0
Jul 23 17:46:47.300843 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.300665    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58894748-nf8v8" event={"ID":"d5c6606c-3e6d-4b9a-8f0e-af41e757c992","Type":"ContainerDied","Data":"5266c1c1c5c89e7e9bbcfbc69d5f0e179277227f45933f511e0a599fd69ac6c3"}
Jul 23 17:46:47.387613 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.387583    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:47.423920 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.423883    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-config\") pod \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") "
Jul 23 17:46:47.424106 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.423955    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-service-ca\") pod \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") "
Jul 23 17:46:47.424106 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424021    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-oauth-serving-cert\") pod \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") "
Jul 23 17:46:47.424106 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424047    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-oauth-config\") pod \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") "
Jul 23 17:46:47.424106 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424075    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-serving-cert\") pod \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") "
Jul 23 17:46:47.424309 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424138    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9928\" (UniqueName: \"kubernetes.io/projected/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-kube-api-access-q9928\") pod \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") "
Jul 23 17:46:47.424309 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424165    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-trusted-ca-bundle\") pod \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\" (UID: \"d5c6606c-3e6d-4b9a-8f0e-af41e757c992\") "
Jul 23 17:46:47.424529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424400    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d5c6606c-3e6d-4b9a-8f0e-af41e757c992" (UID: "d5c6606c-3e6d-4b9a-8f0e-af41e757c992"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:47.424529 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424474    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-service-ca" (OuterVolumeSpecName: "service-ca") pod "d5c6606c-3e6d-4b9a-8f0e-af41e757c992" (UID: "d5c6606c-3e6d-4b9a-8f0e-af41e757c992"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:47.424776 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424739    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d5c6606c-3e6d-4b9a-8f0e-af41e757c992" (UID: "d5c6606c-3e6d-4b9a-8f0e-af41e757c992"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:47.424888 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.424846    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-config" (OuterVolumeSpecName: "console-config") pod "d5c6606c-3e6d-4b9a-8f0e-af41e757c992" (UID: "d5c6606c-3e6d-4b9a-8f0e-af41e757c992"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:47.426721 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.426673    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d5c6606c-3e6d-4b9a-8f0e-af41e757c992" (UID: "d5c6606c-3e6d-4b9a-8f0e-af41e757c992"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:46:47.427156 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.427122    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d5c6606c-3e6d-4b9a-8f0e-af41e757c992" (UID: "d5c6606c-3e6d-4b9a-8f0e-af41e757c992"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:46:47.427311 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.427167    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-kube-api-access-q9928" (OuterVolumeSpecName: "kube-api-access-q9928") pod "d5c6606c-3e6d-4b9a-8f0e-af41e757c992" (UID: "d5c6606c-3e6d-4b9a-8f0e-af41e757c992"). InnerVolumeSpecName "kube-api-access-q9928". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:46:47.525237 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.525193    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9928\" (UniqueName: \"kubernetes.io/projected/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-kube-api-access-q9928\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:47.525237 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.525231    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-trusted-ca-bundle\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:47.525237 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.525242    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:47.525476 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.525253    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-service-ca\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:47.525476 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.525261    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-oauth-serving-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:47.525476 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.525270    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-oauth-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:47.525476 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:47.525278    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5c6606c-3e6d-4b9a-8f0e-af41e757c992-console-serving-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:48.306148 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:48.306113    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f58894748-nf8v8" event={"ID":"d5c6606c-3e6d-4b9a-8f0e-af41e757c992","Type":"ContainerDied","Data":"63cf3b953ebb6eff36a156edf2c5c1dcbf0c9d7efc354aa2021d6e5e5bfb81b0"}
Jul 23 17:46:48.306148 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:48.306147    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f58894748-nf8v8"
Jul 23 17:46:48.306608 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:48.306174    2582 scope.go:117] "RemoveContainer" containerID="5266c1c1c5c89e7e9bbcfbc69d5f0e179277227f45933f511e0a599fd69ac6c3"
Jul 23 17:46:48.321932 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:48.321902    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5f58894748-nf8v8"]
Jul 23 17:46:48.326099 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:48.326035    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5f58894748-nf8v8"]
Jul 23 17:46:49.783958 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:49.783921    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5c6606c-3e6d-4b9a-8f0e-af41e757c992" path="/var/lib/kubelet/pods/d5c6606c-3e6d-4b9a-8f0e-af41e757c992/volumes"
Jul 23 17:46:50.186104 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:50.186073    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-mzb6t"
Jul 23 17:46:57.054284 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.054254    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:57.058095 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.058069    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-59bd5df7f6-vkwhg"
Jul 23 17:46:57.155157 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.155108    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-658757c466-j7jg5" podUID="b8085a58-02dc-40a1-96a0-5f8171fa6aaf" containerName="console" containerID="cri-o://23159b7d834b7c6dcbbf0c924a9c533f7aa7f9d782e37c11f51c94a9da71206b" gracePeriod=15
Jul 23 17:46:57.341828 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.341789    2582 generic.go:356] "Generic (PLEG): container finished" podID="b8085a58-02dc-40a1-96a0-5f8171fa6aaf" containerID="23159b7d834b7c6dcbbf0c924a9c533f7aa7f9d782e37c11f51c94a9da71206b" exitCode=0
Jul 23 17:46:57.342005 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.341857    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658757c466-j7jg5" event={"ID":"b8085a58-02dc-40a1-96a0-5f8171fa6aaf","Type":"ContainerDied","Data":"23159b7d834b7c6dcbbf0c924a9c533f7aa7f9d782e37c11f51c94a9da71206b"}
Jul 23 17:46:57.392168 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.392143    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:57.509082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509047    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-oauth-config\") pod \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") "
Jul 23 17:46:57.509272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509092    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-service-ca\") pod \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") "
Jul 23 17:46:57.509272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509117    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bp872\" (UniqueName: \"kubernetes.io/projected/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-kube-api-access-bp872\") pod \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") "
Jul 23 17:46:57.509272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509140    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-trusted-ca-bundle\") pod \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") "
Jul 23 17:46:57.509272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509175    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-serving-cert\") pod \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") "
Jul 23 17:46:57.509272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509196    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-config\") pod \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") "
Jul 23 17:46:57.509272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509212    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-oauth-serving-cert\") pod \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\" (UID: \"b8085a58-02dc-40a1-96a0-5f8171fa6aaf\") "
Jul 23 17:46:57.509667 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509633    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-service-ca" (OuterVolumeSpecName: "service-ca") pod "b8085a58-02dc-40a1-96a0-5f8171fa6aaf" (UID: "b8085a58-02dc-40a1-96a0-5f8171fa6aaf"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:57.509816 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509661    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "b8085a58-02dc-40a1-96a0-5f8171fa6aaf" (UID: "b8085a58-02dc-40a1-96a0-5f8171fa6aaf"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:57.509816 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509753    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-config" (OuterVolumeSpecName: "console-config") pod "b8085a58-02dc-40a1-96a0-5f8171fa6aaf" (UID: "b8085a58-02dc-40a1-96a0-5f8171fa6aaf"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:57.509816 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.509801    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "b8085a58-02dc-40a1-96a0-5f8171fa6aaf" (UID: "b8085a58-02dc-40a1-96a0-5f8171fa6aaf"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:46:57.511488 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.511459    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "b8085a58-02dc-40a1-96a0-5f8171fa6aaf" (UID: "b8085a58-02dc-40a1-96a0-5f8171fa6aaf"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:46:57.511587 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.511461    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-kube-api-access-bp872" (OuterVolumeSpecName: "kube-api-access-bp872") pod "b8085a58-02dc-40a1-96a0-5f8171fa6aaf" (UID: "b8085a58-02dc-40a1-96a0-5f8171fa6aaf"). InnerVolumeSpecName "kube-api-access-bp872". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:46:57.511587 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.511509    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "b8085a58-02dc-40a1-96a0-5f8171fa6aaf" (UID: "b8085a58-02dc-40a1-96a0-5f8171fa6aaf"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:46:57.610606 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.610525    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-oauth-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:57.610606 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.610553    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-service-ca\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:57.610606 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.610563    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bp872\" (UniqueName: \"kubernetes.io/projected/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-kube-api-access-bp872\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:57.610606 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.610572    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-trusted-ca-bundle\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:57.610606 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.610582    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-serving-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:57.610606 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.610590    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-console-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:57.610606 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:57.610598    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/b8085a58-02dc-40a1-96a0-5f8171fa6aaf-oauth-serving-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:46:58.346769 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:58.346733    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-658757c466-j7jg5" event={"ID":"b8085a58-02dc-40a1-96a0-5f8171fa6aaf","Type":"ContainerDied","Data":"81b131d4d5a7ebd5b78d22d7fe0d0396cc2d542e0cefc02aa3602aba23a426e1"}
Jul 23 17:46:58.346769 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:58.346774    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-658757c466-j7jg5"
Jul 23 17:46:58.347357 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:58.346795    2582 scope.go:117] "RemoveContainer" containerID="23159b7d834b7c6dcbbf0c924a9c533f7aa7f9d782e37c11f51c94a9da71206b"
Jul 23 17:46:58.366294 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:58.366263    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-658757c466-j7jg5"]
Jul 23 17:46:58.370375 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:58.370349    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-658757c466-j7jg5"]
Jul 23 17:46:59.783392 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:46:59.783361    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8085a58-02dc-40a1-96a0-5f8171fa6aaf" path="/var/lib/kubelet/pods/b8085a58-02dc-40a1-96a0-5f8171fa6aaf/volumes"
Jul 23 17:47:18.778233 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:18.778184    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:18.797781 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:18.797757    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:19.436363 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:19.436337    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:29.757088 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.757051    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bb847698d-lv6t8"]
Jul 23 17:47:29.757551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.757307    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5c6606c-3e6d-4b9a-8f0e-af41e757c992" containerName="console"
Jul 23 17:47:29.757551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.757318    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5c6606c-3e6d-4b9a-8f0e-af41e757c992" containerName="console"
Jul 23 17:47:29.757551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.757325    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8085a58-02dc-40a1-96a0-5f8171fa6aaf" containerName="console"
Jul 23 17:47:29.757551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.757331    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8085a58-02dc-40a1-96a0-5f8171fa6aaf" containerName="console"
Jul 23 17:47:29.757551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.757382    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5c6606c-3e6d-4b9a-8f0e-af41e757c992" containerName="console"
Jul 23 17:47:29.757551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.757390    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b8085a58-02dc-40a1-96a0-5f8171fa6aaf" containerName="console"
Jul 23 17:47:29.759285 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.759268    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.763863 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.763843    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 17:47:29.763992 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.763842    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 17:47:29.764189 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.764169    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 17:47:29.764304 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.764190    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 17:47:29.764304 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.764176    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 17:47:29.765394 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.765373    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 17:47:29.765851 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.765588    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 17:47:29.765851 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.765602    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-78r8z\""
Jul 23 17:47:29.768293 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.768277    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 17:47:29.778355 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.778328    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb847698d-lv6t8"]
Jul 23 17:47:29.861095 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.861051    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-service-ca\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.861270 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.861109    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-oauth-config\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.861270 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.861137    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcv4v\" (UniqueName: \"kubernetes.io/projected/fd1eb2cb-2f79-44cf-9697-c3ae68288501-kube-api-access-bcv4v\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.861270 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.861156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-oauth-serving-cert\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.861270 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.861220    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-trusted-ca-bundle\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.861400 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.861280    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-serving-cert\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.861400 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.861306    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-config\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.962464 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.962426    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-trusted-ca-bundle\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.962557 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.962481    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-serving-cert\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.962557 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.962507    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-config\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.962557 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.962526    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-service-ca\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.962665 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.962564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-oauth-config\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.962665 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.962595    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcv4v\" (UniqueName: \"kubernetes.io/projected/fd1eb2cb-2f79-44cf-9697-c3ae68288501-kube-api-access-bcv4v\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.962860 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.962757    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-oauth-serving-cert\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.963382 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.963354    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-config\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.963564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.963406    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-service-ca\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.963564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.963430    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-oauth-serving-cert\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.963694 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.963658    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-trusted-ca-bundle\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.965100 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.965076    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-oauth-config\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.965218 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.965201    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-serving-cert\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:29.972239 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:29.972214    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcv4v\" (UniqueName: \"kubernetes.io/projected/fd1eb2cb-2f79-44cf-9697-c3ae68288501-kube-api-access-bcv4v\") pod \"console-bb847698d-lv6t8\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") " pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:30.068585 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:30.068547    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:30.202393 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:30.202367    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bb847698d-lv6t8"]
Jul 23 17:47:30.204929 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:47:30.204902    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1eb2cb_2f79_44cf_9697_c3ae68288501.slice/crio-a2bb8892b806681e7ee681678f71bb421e508b1c0d4b0bf5012b238b981f87c9 WatchSource:0}: Error finding container a2bb8892b806681e7ee681678f71bb421e508b1c0d4b0bf5012b238b981f87c9: Status 404 returned error can't find the container with id a2bb8892b806681e7ee681678f71bb421e508b1c0d4b0bf5012b238b981f87c9
Jul 23 17:47:30.458607 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:30.458506    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb847698d-lv6t8" event={"ID":"fd1eb2cb-2f79-44cf-9697-c3ae68288501","Type":"ContainerStarted","Data":"01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51"}
Jul 23 17:47:30.458607 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:30.458566    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb847698d-lv6t8" event={"ID":"fd1eb2cb-2f79-44cf-9697-c3ae68288501","Type":"ContainerStarted","Data":"a2bb8892b806681e7ee681678f71bb421e508b1c0d4b0bf5012b238b981f87c9"}
Jul 23 17:47:30.479876 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:30.479820    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bb847698d-lv6t8" podStartSLOduration=1.479800086 podStartE2EDuration="1.479800086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:47:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:47:30.47864899 +0000 UTC m=+145.318156938" watchObservedRunningTime="2026-07-23 17:47:30.479800086 +0000 UTC m=+145.319308038"
Jul 23 17:47:36.879876 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:36.879834    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:47:36.880451 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:36.880237    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="prometheus" containerID="cri-o://d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e" gracePeriod=600
Jul 23 17:47:36.880611 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:36.880586    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy" containerID="cri-o://ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a" gracePeriod=600
Jul 23 17:47:36.880888 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:36.880865    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-thanos" containerID="cri-o://287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c" gracePeriod=600
Jul 23 17:47:36.881094 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:36.880619    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="thanos-sidecar" containerID="cri-o://02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133" gracePeriod=600
Jul 23 17:47:36.881239 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:36.880637    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="config-reloader" containerID="cri-o://9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448" gracePeriod=600
Jul 23 17:47:36.882015 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:36.880598    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-web" containerID="cri-o://8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b" gracePeriod=600
Jul 23 17:47:37.136130 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.136058    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.327996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.327947    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-tls\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328012    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-metrics-client-certs\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328034    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-web-config\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328056    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-db\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328075    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-kube-rbac-proxy\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328095    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328118    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-trusted-ca-bundle\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328154    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kqvk7\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-kube-api-access-kqvk7\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328178    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-thanos-prometheus-http-client-file\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328194    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-1\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328207 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328211    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-grpc-tls\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328230    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-serving-certs-ca-bundle\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328248    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-2\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328269    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-tls-assets\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328301    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328329    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-config\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328385    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-config-out\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328415    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-0\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328442    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-kubelet-serving-ca-bundle\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328474    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-metrics-client-ca\") pod \"c48c5570-8627-43d4-9a32-581493a2a345\" (UID: \"c48c5570-8627-43d4-9a32-581493a2a345\") "
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328576    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:47:37.328689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328618    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:47:37.329214 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328785    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.329214 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.328804    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.329311 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.329269    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 17:47:37.329311 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.329295    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:47:37.329996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.329652    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:47:37.329996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.329738    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:47:37.329996 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.329968    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:47:37.330755 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.330724    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:47:37.332395 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.332353    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:47:37.332892 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.332860    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.333128 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333066    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.333350 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333210    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-config-out" (OuterVolumeSpecName: "config-out") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 17:47:37.333350 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333276    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-kube-api-access-kqvk7" (OuterVolumeSpecName: "kube-api-access-kqvk7") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "kube-api-access-kqvk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:47:37.333350 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333283    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.333546 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333363    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.333546 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333432    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.333546 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333502    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.333546 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.333527    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-config" (OuterVolumeSpecName: "config") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.334311 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.334292    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.342083 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.342053    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-web-config" (OuterVolumeSpecName: "web-config") pod "c48c5570-8627-43d4-9a32-581493a2a345" (UID: "c48c5570-8627-43d4-9a32-581493a2a345"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429439    2582 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-config-out\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429472    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429483    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429492    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-metrics-client-ca\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429503    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-tls\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429511    2582 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-metrics-client-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429520    2582 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-web-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429531    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-k8s-db\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429540    2582 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-kube-rbac-proxy\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429549    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429558    2582 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-prometheus-trusted-ca-bundle\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429566    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqvk7\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-kube-api-access-kqvk7\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429575    2582 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-thanos-prometheus-http-client-file\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429583    2582 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-grpc-tls\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429591    2582 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c48c5570-8627-43d4-9a32-581493a2a345-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429599    2582 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c48c5570-8627-43d4-9a32-581493a2a345-tls-assets\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429607    2582 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.429916 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.429616    2582 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c48c5570-8627-43d4-9a32-581493a2a345-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:47:37.490444 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490414    2582 generic.go:356] "Generic (PLEG): container finished" podID="c48c5570-8627-43d4-9a32-581493a2a345" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c" exitCode=0
Jul 23 17:47:37.490444 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490438    2582 generic.go:356] "Generic (PLEG): container finished" podID="c48c5570-8627-43d4-9a32-581493a2a345" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a" exitCode=0
Jul 23 17:47:37.490444 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490445    2582 generic.go:356] "Generic (PLEG): container finished" podID="c48c5570-8627-43d4-9a32-581493a2a345" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b" exitCode=0
Jul 23 17:47:37.490444 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490451    2582 generic.go:356] "Generic (PLEG): container finished" podID="c48c5570-8627-43d4-9a32-581493a2a345" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133" exitCode=0
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490457    2582 generic.go:356] "Generic (PLEG): container finished" podID="c48c5570-8627-43d4-9a32-581493a2a345" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448" exitCode=0
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490462    2582 generic.go:356] "Generic (PLEG): container finished" podID="c48c5570-8627-43d4-9a32-581493a2a345" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e" exitCode=0
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490493    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490505    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"}
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490593    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"}
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490639    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"}
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490657    2582 scope.go:117] "RemoveContainer" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.490689 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490680    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"}
Jul 23 17:47:37.491009 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490741    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"}
Jul 23 17:47:37.491009 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490784    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"}
Jul 23 17:47:37.491009 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.490825    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c48c5570-8627-43d4-9a32-581493a2a345","Type":"ContainerDied","Data":"12a102549777fbbf6f52aeb5b15c9969e537663ef6ddb3af91a69791714b129d"}
Jul 23 17:47:37.498958 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.498939    2582 scope.go:117] "RemoveContainer" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.505968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.505948    2582 scope.go:117] "RemoveContainer" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.513439 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.513350    2582 scope.go:117] "RemoveContainer" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.516042 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.516021    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:47:37.521311 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.521288    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:47:37.522229 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.522213    2582 scope.go:117] "RemoveContainer" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.531223 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.531204    2582 scope.go:117] "RemoveContainer" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.538832 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.538810    2582 scope.go:117] "RemoveContainer" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.546289 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.546267    2582 scope.go:117] "RemoveContainer" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.546591 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:47:37.546569    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": container with ID starting with 287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c not found: ID does not exist" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.546634 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.546603    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"} err="failed to get container status \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": rpc error: code = NotFound desc = could not find container \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": container with ID starting with 287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c not found: ID does not exist"
Jul 23 17:47:37.546674 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.546639    2582 scope.go:117] "RemoveContainer" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.546941 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:47:37.546923    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": container with ID starting with ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a not found: ID does not exist" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.546993 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.546947    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"} err="failed to get container status \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": rpc error: code = NotFound desc = could not find container \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": container with ID starting with ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a not found: ID does not exist"
Jul 23 17:47:37.546993 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.546964    2582 scope.go:117] "RemoveContainer" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.547192 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:47:37.547177    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": container with ID starting with 8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b not found: ID does not exist" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.547236 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547195    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"} err="failed to get container status \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": rpc error: code = NotFound desc = could not find container \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": container with ID starting with 8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b not found: ID does not exist"
Jul 23 17:47:37.547236 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547211    2582 scope.go:117] "RemoveContainer" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.547437 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:47:37.547412    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": container with ID starting with 02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133 not found: ID does not exist" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.547541 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547440    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"} err="failed to get container status \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": rpc error: code = NotFound desc = could not find container \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": container with ID starting with 02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133 not found: ID does not exist"
Jul 23 17:47:37.547541 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547456    2582 scope.go:117] "RemoveContainer" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.547732 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:47:37.547697    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": container with ID starting with 9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448 not found: ID does not exist" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.547784 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547739    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"} err="failed to get container status \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": rpc error: code = NotFound desc = could not find container \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": container with ID starting with 9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448 not found: ID does not exist"
Jul 23 17:47:37.547784 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547755    2582 scope.go:117] "RemoveContainer" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.547978 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:47:37.547963    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": container with ID starting with d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e not found: ID does not exist" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.548020 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547980    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"} err="failed to get container status \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": rpc error: code = NotFound desc = could not find container \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": container with ID starting with d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e not found: ID does not exist"
Jul 23 17:47:37.548020 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.547993    2582 scope.go:117] "RemoveContainer" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.548226 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:47:37.548212    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": container with ID starting with 65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a not found: ID does not exist" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.548268 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548230    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"} err="failed to get container status \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": rpc error: code = NotFound desc = could not find container \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": container with ID starting with 65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a not found: ID does not exist"
Jul 23 17:47:37.548268 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548243    2582 scope.go:117] "RemoveContainer" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.548437 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548419    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"} err="failed to get container status \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": rpc error: code = NotFound desc = could not find container \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": container with ID starting with 287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c not found: ID does not exist"
Jul 23 17:47:37.548474 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548438    2582 scope.go:117] "RemoveContainer" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.548634 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548618    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"} err="failed to get container status \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": rpc error: code = NotFound desc = could not find container \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": container with ID starting with ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a not found: ID does not exist"
Jul 23 17:47:37.548634 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548634    2582 scope.go:117] "RemoveContainer" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.548824 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548806    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"} err="failed to get container status \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": rpc error: code = NotFound desc = could not find container \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": container with ID starting with 8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b not found: ID does not exist"
Jul 23 17:47:37.548824 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.548822    2582 scope.go:117] "RemoveContainer" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.549022 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549005    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"} err="failed to get container status \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": rpc error: code = NotFound desc = could not find container \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": container with ID starting with 02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133 not found: ID does not exist"
Jul 23 17:47:37.549022 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549022    2582 scope.go:117] "RemoveContainer" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.549213 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549198    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"} err="failed to get container status \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": rpc error: code = NotFound desc = could not find container \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": container with ID starting with 9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448 not found: ID does not exist"
Jul 23 17:47:37.549213 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549212    2582 scope.go:117] "RemoveContainer" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.549401 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549387    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"} err="failed to get container status \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": rpc error: code = NotFound desc = could not find container \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": container with ID starting with d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e not found: ID does not exist"
Jul 23 17:47:37.549440 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549401    2582 scope.go:117] "RemoveContainer" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.549578 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549563    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"} err="failed to get container status \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": rpc error: code = NotFound desc = could not find container \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": container with ID starting with 65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a not found: ID does not exist"
Jul 23 17:47:37.549578 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549578    2582 scope.go:117] "RemoveContainer" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.549805 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549790    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"} err="failed to get container status \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": rpc error: code = NotFound desc = could not find container \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": container with ID starting with 287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c not found: ID does not exist"
Jul 23 17:47:37.549845 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549805    2582 scope.go:117] "RemoveContainer" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.549998 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549983    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"} err="failed to get container status \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": rpc error: code = NotFound desc = could not find container \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": container with ID starting with ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a not found: ID does not exist"
Jul 23 17:47:37.550043 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.549998    2582 scope.go:117] "RemoveContainer" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.550176 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.550160    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"} err="failed to get container status \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": rpc error: code = NotFound desc = could not find container \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": container with ID starting with 8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b not found: ID does not exist"
Jul 23 17:47:37.550213 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.550175    2582 scope.go:117] "RemoveContainer" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.550399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.550339    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"} err="failed to get container status \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": rpc error: code = NotFound desc = could not find container \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": container with ID starting with 02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133 not found: ID does not exist"
Jul 23 17:47:37.550460 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.550400    2582 scope.go:117] "RemoveContainer" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.550971 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.550946    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"} err="failed to get container status \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": rpc error: code = NotFound desc = could not find container \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": container with ID starting with 9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448 not found: ID does not exist"
Jul 23 17:47:37.550971 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.550972    2582 scope.go:117] "RemoveContainer" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.551378 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.551347    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"} err="failed to get container status \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": rpc error: code = NotFound desc = could not find container \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": container with ID starting with d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e not found: ID does not exist"
Jul 23 17:47:37.551378 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.551376    2582 scope.go:117] "RemoveContainer" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.551620 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.551599    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"} err="failed to get container status \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": rpc error: code = NotFound desc = could not find container \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": container with ID starting with 65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a not found: ID does not exist"
Jul 23 17:47:37.551752 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.551622    2582 scope.go:117] "RemoveContainer" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.551877 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.551860    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"} err="failed to get container status \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": rpc error: code = NotFound desc = could not find container \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": container with ID starting with 287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c not found: ID does not exist"
Jul 23 17:47:37.551927 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.551878    2582 scope.go:117] "RemoveContainer" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.552118 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552101    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"} err="failed to get container status \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": rpc error: code = NotFound desc = could not find container \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": container with ID starting with ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a not found: ID does not exist"
Jul 23 17:47:37.552163 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552119    2582 scope.go:117] "RemoveContainer" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.552348 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552327    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"} err="failed to get container status \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": rpc error: code = NotFound desc = could not find container \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": container with ID starting with 8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b not found: ID does not exist"
Jul 23 17:47:37.552394 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552352    2582 scope.go:117] "RemoveContainer" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.552566 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552549    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"} err="failed to get container status \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": rpc error: code = NotFound desc = could not find container \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": container with ID starting with 02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133 not found: ID does not exist"
Jul 23 17:47:37.552605 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552566    2582 scope.go:117] "RemoveContainer" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.552799 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552781    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"} err="failed to get container status \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": rpc error: code = NotFound desc = could not find container \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": container with ID starting with 9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448 not found: ID does not exist"
Jul 23 17:47:37.552876 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552802    2582 scope.go:117] "RemoveContainer" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.552876 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552806    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:47:37.553007 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.552991    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"} err="failed to get container status \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": rpc error: code = NotFound desc = could not find container \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": container with ID starting with d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e not found: ID does not exist"
Jul 23 17:47:37.553069 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553010    2582 scope.go:117] "RemoveContainer" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.553108 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553072    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="init-config-reloader"
Jul 23 17:47:37.553108 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553086    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="init-config-reloader"
Jul 23 17:47:37.553108 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553099    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="config-reloader"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553109    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="config-reloader"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553119    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-web"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553128    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-web"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553137    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553146    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553155    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-thanos"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553164    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-thanos"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553174    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="thanos-sidecar"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553182    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="thanos-sidecar"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553203    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="prometheus"
Jul 23 17:47:37.553215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553209    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="prometheus"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553247    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"} err="failed to get container status \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": rpc error: code = NotFound desc = could not find container \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": container with ID starting with 65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a not found: ID does not exist"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553268    2582 scope.go:117] "RemoveContainer" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553257    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="thanos-sidecar"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553310    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="prometheus"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553318    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-thanos"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553326    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="config-reloader"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553339    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy-web"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553346    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c48c5570-8627-43d4-9a32-581493a2a345" containerName="kube-rbac-proxy"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553474    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"} err="failed to get container status \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": rpc error: code = NotFound desc = could not find container \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": container with ID starting with 287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c not found: ID does not exist"
Jul 23 17:47:37.553564 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553489    2582 scope.go:117] "RemoveContainer" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.553981 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553692    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"} err="failed to get container status \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": rpc error: code = NotFound desc = could not find container \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": container with ID starting with ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a not found: ID does not exist"
Jul 23 17:47:37.553981 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.553731    2582 scope.go:117] "RemoveContainer" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.554060 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554000    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"} err="failed to get container status \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": rpc error: code = NotFound desc = could not find container \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": container with ID starting with 8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b not found: ID does not exist"
Jul 23 17:47:37.554060 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554018    2582 scope.go:117] "RemoveContainer" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.554232 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554214    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"} err="failed to get container status \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": rpc error: code = NotFound desc = could not find container \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": container with ID starting with 02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133 not found: ID does not exist"
Jul 23 17:47:37.554298 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554235    2582 scope.go:117] "RemoveContainer" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.554461 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554439    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"} err="failed to get container status \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": rpc error: code = NotFound desc = could not find container \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": container with ID starting with 9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448 not found: ID does not exist"
Jul 23 17:47:37.554505 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554465    2582 scope.go:117] "RemoveContainer" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.554657 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554640    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"} err="failed to get container status \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": rpc error: code = NotFound desc = could not find container \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": container with ID starting with d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e not found: ID does not exist"
Jul 23 17:47:37.554740 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554659    2582 scope.go:117] "RemoveContainer" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.554879 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554864    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"} err="failed to get container status \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": rpc error: code = NotFound desc = could not find container \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": container with ID starting with 65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a not found: ID does not exist"
Jul 23 17:47:37.554947 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.554881    2582 scope.go:117] "RemoveContainer" containerID="287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"
Jul 23 17:47:37.555115 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555098    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c"} err="failed to get container status \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": rpc error: code = NotFound desc = could not find container \"287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c\": container with ID starting with 287df15bdbe405b52f6aba64a82d08961a9d06767f6a0825c7a87e3c6ea2335c not found: ID does not exist"
Jul 23 17:47:37.555172 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555115    2582 scope.go:117] "RemoveContainer" containerID="ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"
Jul 23 17:47:37.555323 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555304    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a"} err="failed to get container status \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": rpc error: code = NotFound desc = could not find container \"ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a\": container with ID starting with ee96e67f53738594208acd7eb5f0df8be7c660dc8aeaa69f33a2c36bc3f90b9a not found: ID does not exist"
Jul 23 17:47:37.555384 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555323    2582 scope.go:117] "RemoveContainer" containerID="8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"
Jul 23 17:47:37.555540 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555522    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b"} err="failed to get container status \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": rpc error: code = NotFound desc = could not find container \"8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b\": container with ID starting with 8dc991591a276f0815fc6c44289bb5b4789d87959eb8b607664040a8b6b2773b not found: ID does not exist"
Jul 23 17:47:37.555584 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555540    2582 scope.go:117] "RemoveContainer" containerID="02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"
Jul 23 17:47:37.555738 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555701    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133"} err="failed to get container status \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": rpc error: code = NotFound desc = could not find container \"02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133\": container with ID starting with 02caa21a3c83a9654b3322889e9b78b95af2a66c2f0d1ad4a29244699e5ab133 not found: ID does not exist"
Jul 23 17:47:37.555786 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555739    2582 scope.go:117] "RemoveContainer" containerID="9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"
Jul 23 17:47:37.555924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555907    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448"} err="failed to get container status \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": rpc error: code = NotFound desc = could not find container \"9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448\": container with ID starting with 9e3f149a857cd8189b6c26b3990d7698ba2551a5b7338d2aacb4b9b898116448 not found: ID does not exist"
Jul 23 17:47:37.555967 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.555925    2582 scope.go:117] "RemoveContainer" containerID="d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"
Jul 23 17:47:37.556123 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.556108    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e"} err="failed to get container status \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": rpc error: code = NotFound desc = could not find container \"d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e\": container with ID starting with d49adb9d381b3764244302d377a40041064523932770ca16bfa84230e103fe1e not found: ID does not exist"
Jul 23 17:47:37.556164 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.556123    2582 scope.go:117] "RemoveContainer" containerID="65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"
Jul 23 17:47:37.556298 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.556279    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a"} err="failed to get container status \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": rpc error: code = NotFound desc = could not find container \"65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a\": container with ID starting with 65bbee6872152c613934edac2ac3ab3724d3465b4778da546372e73e84f0d24a not found: ID does not exist"
Jul 23 17:47:37.556935 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.556921    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.559836 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.559820    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 23 17:47:37.560087 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560073    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 23 17:47:37.560151 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560086    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-bb6tx\""
Jul 23 17:47:37.560271 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560251    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-c391mm8sv2ilj\""
Jul 23 17:47:37.560271 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560262    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 23 17:47:37.560390 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560294    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 23 17:47:37.560445 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560354    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 23 17:47:37.560593 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560576    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 23 17:47:37.560656 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560625    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 23 17:47:37.560656 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.560627    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 23 17:47:37.561042 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.561027    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 23 17:47:37.561319 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.561300    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 23 17:47:37.561412 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.561300    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 23 17:47:37.561412 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.561299    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 23 17:47:37.570836 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.570808    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 23 17:47:37.571676 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.571659    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 23 17:47:37.579215 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.579188    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:47:37.732511 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732409    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732511 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732451    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-config\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732511 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732468    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732511 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732493    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732511 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732510    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732559    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732595    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732621    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732671    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732700    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732752    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732772    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx456\" (UniqueName: \"kubernetes.io/projected/084afdb7-8a71-4b96-a774-9487ce0ffd9e-kube-api-access-bx456\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732803    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.732899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732851    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.733251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732907    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.733251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732942    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/084afdb7-8a71-4b96-a774-9487ce0ffd9e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.733251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732966    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.733251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.732997    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/084afdb7-8a71-4b96-a774-9487ce0ffd9e-config-out\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.733251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.733018    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-web-config\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.733251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.733058    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.782973 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.782940    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c48c5570-8627-43d4-9a32-581493a2a345" path="/var/lib/kubelet/pods/c48c5570-8627-43d4-9a32-581493a2a345/volumes"
Jul 23 17:47:37.833568 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833523    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/084afdb7-8a71-4b96-a774-9487ce0ffd9e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833568 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833780 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833594    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/084afdb7-8a71-4b96-a774-9487ce0ffd9e-config-out\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833780 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833611    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-web-config\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833780 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833661    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833780 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833750    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833984 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833806    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-config\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833984 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833831    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833984 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833864    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833984 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833893    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833984 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833935    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.833984 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.833960    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834006    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834030    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834051    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834085    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834114    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx456\" (UniqueName: \"kubernetes.io/projected/084afdb7-8a71-4b96-a774-9487ce0ffd9e-kube-api-access-bx456\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834140    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834167    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834280 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834224    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834655 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834452    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834655 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834559    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.834789 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.834662    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.835087 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.835046    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.836462 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.835598    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.836462 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.835685    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.837154 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.836830    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/084afdb7-8a71-4b96-a774-9487ce0ffd9e-config-out\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.837154 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.836872    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.837154 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.836944    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-web-config\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.837154 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.836973    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.837403 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.837222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-config\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.837403 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.837225    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.837630 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.837603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/084afdb7-8a71-4b96-a774-9487ce0ffd9e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.838307 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.838289    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.839369 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.839347    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.839450 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.839368    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.839544 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.839526    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.839785 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.839766    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/084afdb7-8a71-4b96-a774-9487ce0ffd9e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.839905 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.839888    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/084afdb7-8a71-4b96-a774-9487ce0ffd9e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.846656 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.846636    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx456\" (UniqueName: \"kubernetes.io/projected/084afdb7-8a71-4b96-a774-9487ce0ffd9e-kube-api-access-bx456\") pod \"prometheus-k8s-0\" (UID: \"084afdb7-8a71-4b96-a774-9487ce0ffd9e\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:37.869651 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:37.869612    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:47:38.041013 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:38.040929    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 23 17:47:38.044218 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:47:38.044193    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod084afdb7_8a71_4b96_a774_9487ce0ffd9e.slice/crio-93e45dac1d488f5bc830f0ce318e598ecbe5f3f73614eecae544fd1835801768 WatchSource:0}: Error finding container 93e45dac1d488f5bc830f0ce318e598ecbe5f3f73614eecae544fd1835801768: Status 404 returned error can't find the container with id 93e45dac1d488f5bc830f0ce318e598ecbe5f3f73614eecae544fd1835801768
Jul 23 17:47:38.494784 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:38.494753    2582 generic.go:356] "Generic (PLEG): container finished" podID="084afdb7-8a71-4b96-a774-9487ce0ffd9e" containerID="48d44b18e665bec26ddbb8c3182184b67f4f1e1d219c8fb01c1c42c1bc782278" exitCode=0
Jul 23 17:47:38.494982 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:38.494847    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerDied","Data":"48d44b18e665bec26ddbb8c3182184b67f4f1e1d219c8fb01c1c42c1bc782278"}
Jul 23 17:47:38.494982 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:38.494921    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerStarted","Data":"93e45dac1d488f5bc830f0ce318e598ecbe5f3f73614eecae544fd1835801768"}
Jul 23 17:47:39.502817 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:39.502781    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerStarted","Data":"6facb6c6e93b380ff3851ecbb9de412e7edefc888dbfe323f0e41f7547ba72b0"}
Jul 23 17:47:39.503192 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:39.502874    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerStarted","Data":"320ee9ea3dddacc47b6d93ba87fcda1aa09e11ece89b5d3c9d2e9e3343e1c1c1"}
Jul 23 17:47:39.503192 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:39.502903    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerStarted","Data":"e050cb259eae9548317f4765c46d0b53783a172aae8ce8acaa14d2cfe9bab314"}
Jul 23 17:47:39.503192 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:39.502928    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerStarted","Data":"a79693b96d56bd9703c5fb15f69714c1cd95d604e0b22a5443739facfe86ffac"}
Jul 23 17:47:39.503192 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:39.502953    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerStarted","Data":"203f2c181eb34607e9fa29d6a536fc675b750e2f25d047ea7e8da99f219cf87b"}
Jul 23 17:47:39.503192 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:39.502976    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"084afdb7-8a71-4b96-a774-9487ce0ffd9e","Type":"ContainerStarted","Data":"90a7a71b9295adf7e46f3940db54290fa1133263dff5a723558360947b0058be"}
Jul 23 17:47:39.540732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:39.540649    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.520726262 podStartE2EDuration="2.540630131s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.903869ms" isStatefulPod=false podCreationTimestamp="2026-07-23 17:47:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:47:39.53778874 +0000 UTC m=+154.377296688" watchObservedRunningTime="2026-07-23 17:47:39.540630131 +0000 UTC m=+154.380138078"
Jul 23 17:47:40.069462 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:40.069423    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:40.069636 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:40.069493    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:40.074073 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:40.074051    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:40.511740 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:40.511651    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:47:42.870562 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:47:42.870511    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:48:37.870275 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:48:37.870232    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:48:37.886469 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:48:37.886436    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:48:38.728387 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:48:38.728358    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 23 17:49:39.224833 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.224753    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"]
Jul 23 17:49:39.227571 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.227555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.230253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.230225    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 17:49:39.231130 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.231107    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-bwzdx\""
Jul 23 17:49:39.231216 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.231110    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 17:49:39.236477 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.236455    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"]
Jul 23 17:49:39.358586 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.358548    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59aee851-756c-4d70-9855-90ea91354016-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gstdx\" (UID: \"59aee851-756c-4d70-9855-90ea91354016\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.358772 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.358603    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnkwb\" (UniqueName: \"kubernetes.io/projected/59aee851-756c-4d70-9855-90ea91354016-kube-api-access-bnkwb\") pod \"openshift-lws-operator-fd8ccff4c-gstdx\" (UID: \"59aee851-756c-4d70-9855-90ea91354016\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.459924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.459881    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59aee851-756c-4d70-9855-90ea91354016-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gstdx\" (UID: \"59aee851-756c-4d70-9855-90ea91354016\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.460091 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.459945    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnkwb\" (UniqueName: \"kubernetes.io/projected/59aee851-756c-4d70-9855-90ea91354016-kube-api-access-bnkwb\") pod \"openshift-lws-operator-fd8ccff4c-gstdx\" (UID: \"59aee851-756c-4d70-9855-90ea91354016\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.460286 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.460267    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59aee851-756c-4d70-9855-90ea91354016-tmp\") pod \"openshift-lws-operator-fd8ccff4c-gstdx\" (UID: \"59aee851-756c-4d70-9855-90ea91354016\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.469373 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.469344    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnkwb\" (UniqueName: \"kubernetes.io/projected/59aee851-756c-4d70-9855-90ea91354016-kube-api-access-bnkwb\") pod \"openshift-lws-operator-fd8ccff4c-gstdx\" (UID: \"59aee851-756c-4d70-9855-90ea91354016\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.550198 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.550116    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"
Jul 23 17:49:39.693382 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.693350    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx"]
Jul 23 17:49:39.696701 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:49:39.696672    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod59aee851_756c_4d70_9855_90ea91354016.slice/crio-b5f934eb256b6a3a599c592230b75751543d5f114ffad5bf465f6d29b41c35ee WatchSource:0}: Error finding container b5f934eb256b6a3a599c592230b75751543d5f114ffad5bf465f6d29b41c35ee: Status 404 returned error can't find the container with id b5f934eb256b6a3a599c592230b75751543d5f114ffad5bf465f6d29b41c35ee
Jul 23 17:49:39.933745 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:39.933698    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx" event={"ID":"59aee851-756c-4d70-9855-90ea91354016","Type":"ContainerStarted","Data":"b5f934eb256b6a3a599c592230b75751543d5f114ffad5bf465f6d29b41c35ee"}
Jul 23 17:49:42.949807 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:42.949767    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx" event={"ID":"59aee851-756c-4d70-9855-90ea91354016","Type":"ContainerStarted","Data":"aea31fbf46a96653f409eaad38c17b313e205c5168fda310029606c17adfa612"}
Jul 23 17:49:42.972352 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:49:42.972294    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-gstdx" podStartSLOduration=1.433977021 podStartE2EDuration="3.97227891s" totalImagesPullingTime="2.538301889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:49:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:49:42.970209494 +0000 UTC m=+277.809717443" watchObservedRunningTime="2026-07-23 17:49:42.97227891 +0000 UTC m=+277.811786858"
Jul 23 17:50:02.656123 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.656084    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"]
Jul 23 17:50:02.659292 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.659274    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.661743 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.661700    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-tmp\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.661868 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.661769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c8dp8\" (UniqueName: \"kubernetes.io/projected/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-kube-api-access-c8dp8\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.661911 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.661874    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.661963 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.661945    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-webhook-cert\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.662016 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.661979    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-manifests\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.662181 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.662164    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 17:50:02.662238 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.662177    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-25pnk\""
Jul 23 17:50:02.662443 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.662427    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 17:50:02.662544 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.662525    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 17:50:02.662601 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.662548    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 17:50:02.678911 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.678881    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"]
Jul 23 17:50:02.762610 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.762571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.762829 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.762624    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-webhook-cert\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.762829 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.762642    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-manifests\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.762829 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.762663    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-tmp\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.762829 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.762736    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c8dp8\" (UniqueName: \"kubernetes.io/projected/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-kube-api-access-c8dp8\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.763185 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.763159    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-manifests\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.765125 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.765100    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-tmp\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.765186 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.765156    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-webhook-cert\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.765352 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.765332    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-apiservice-cert\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.775200 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.775163    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c8dp8\" (UniqueName: \"kubernetes.io/projected/5dcac08f-5065-4dfa-bd0a-d5ccba92c976-kube-api-access-c8dp8\") pod \"opendatahub-operator-controller-manager-547c58d5d9-hbh8z\" (UID: \"5dcac08f-5065-4dfa-bd0a-d5ccba92c976\") " pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:02.974114 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:02.974019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:03.116734 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:03.116686    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"]
Jul 23 17:50:03.119007 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:50:03.118980    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dcac08f_5065_4dfa_bd0a_d5ccba92c976.slice/crio-34ef3de89e8476a1735a68af276660959b23157e62f9a4294339bffb089ea9ee WatchSource:0}: Error finding container 34ef3de89e8476a1735a68af276660959b23157e62f9a4294339bffb089ea9ee: Status 404 returned error can't find the container with id 34ef3de89e8476a1735a68af276660959b23157e62f9a4294339bffb089ea9ee
Jul 23 17:50:04.031275 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:04.031240    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z" event={"ID":"5dcac08f-5065-4dfa-bd0a-d5ccba92c976","Type":"ContainerStarted","Data":"34ef3de89e8476a1735a68af276660959b23157e62f9a4294339bffb089ea9ee"}
Jul 23 17:50:06.270613 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:06.270579    2582 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 17:50:07.044849 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:07.044809    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z" event={"ID":"5dcac08f-5065-4dfa-bd0a-d5ccba92c976","Type":"ContainerStarted","Data":"b5d8c8d97987c0a697e6d65a6b070ed3ca6b84fcdaaaed51c3497ff21f3eff01"}
Jul 23 17:50:08.050625 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:08.050585    2582 generic.go:356] "Generic (PLEG): container finished" podID="5dcac08f-5065-4dfa-bd0a-d5ccba92c976" containerID="b5d8c8d97987c0a697e6d65a6b070ed3ca6b84fcdaaaed51c3497ff21f3eff01" exitCode=0
Jul 23 17:50:08.051059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:08.050632    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z" event={"ID":"5dcac08f-5065-4dfa-bd0a-d5ccba92c976","Type":"ContainerDied","Data":"b5d8c8d97987c0a697e6d65a6b070ed3ca6b84fcdaaaed51c3497ff21f3eff01"}
Jul 23 17:50:08.051182 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:08.051167    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 17:50:09.056627 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:09.056589    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z" event={"ID":"5dcac08f-5065-4dfa-bd0a-d5ccba92c976","Type":"ContainerStarted","Data":"7a22127320f69bcbd62b8e856480ce6dfc8b1a2fd131d2bddb0b8c0d03df004a"}
Jul 23 17:50:09.057035 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:09.056727    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:09.080964 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:09.080913    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z" podStartSLOduration=2.482489045 podStartE2EDuration="7.08089061s" totalImagesPullingTime="3.531248202s" totalInitContainerRuntime="1.067153363s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:50:02 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:50:09.079477829 +0000 UTC m=+303.918985801" watchObservedRunningTime="2026-07-23 17:50:09.08089061 +0000 UTC m=+303.920398562"
Jul 23 17:50:20.064085 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:20.064052    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-547c58d5d9-hbh8z"
Jul 23 17:50:22.580607 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.580566    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"]
Jul 23 17:50:22.587341 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.587311    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.593107 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.593076    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ttd59\""
Jul 23 17:50:22.593303 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.593285    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 23 17:50:22.594208 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.594173    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 17:50:22.594208 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.594198    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 23 17:50:22.594438 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.594181    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 17:50:22.606412 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.606385    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"]
Jul 23 17:50:22.631429 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.631399    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tls-certs\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.631609 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.631446    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tmp\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.631609 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.631471    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzmlt\" (UniqueName: \"kubernetes.io/projected/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-kube-api-access-jzmlt\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.732951 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.732910    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzmlt\" (UniqueName: \"kubernetes.io/projected/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-kube-api-access-jzmlt\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.733132 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.732987    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tls-certs\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.733132 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.733021    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tmp\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.733216 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:50:22.733145    2582 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 23 17:50:22.733251 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:50:22.733221    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tls-certs podName:f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2 nodeName:}" failed. No retries permitted until 2026-07-23 17:50:23.233200812 +0000 UTC m=+318.072708761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tls-certs") pod "kube-auth-proxy-5b6c5d7974-k9nj7" (UID: "f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2") : secret "kube-auth-proxy-tls" not found
Jul 23 17:50:22.735208 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.735186    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tmp\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:22.752263 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:22.752226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzmlt\" (UniqueName: \"kubernetes.io/projected/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-kube-api-access-jzmlt\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:23.237060 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:23.237017    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tls-certs\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:23.239523 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:23.239492    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2-tls-certs\") pod \"kube-auth-proxy-5b6c5d7974-k9nj7\" (UID: \"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2\") " pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:23.497075 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:23.496971    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"
Jul 23 17:50:23.645390 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:23.645261    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7"]
Jul 23 17:50:23.648278 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:50:23.648250    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3d89cb6_c9b2_4f7c_99c5_068e2b30fbd2.slice/crio-6a6c0b3bae3451dce5368e87fa2452b06c5693c1e0af6bdc3cdff8024ba926c2 WatchSource:0}: Error finding container 6a6c0b3bae3451dce5368e87fa2452b06c5693c1e0af6bdc3cdff8024ba926c2: Status 404 returned error can't find the container with id 6a6c0b3bae3451dce5368e87fa2452b06c5693c1e0af6bdc3cdff8024ba926c2
Jul 23 17:50:24.051074 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.051039    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9xks5"]
Jul 23 17:50:24.054259 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.054240    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.057988 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.057964    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 23 17:50:24.058924 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.058903    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-fjjks\""
Jul 23 17:50:24.070288 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.070224    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9xks5"]
Jul 23 17:50:24.117191 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.117160    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7" event={"ID":"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2","Type":"ContainerStarted","Data":"6a6c0b3bae3451dce5368e87fa2452b06c5693c1e0af6bdc3cdff8024ba926c2"}
Jul 23 17:50:24.145583 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.145549    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785-tls-certs\") pod \"model-serving-api-54d89bd5fc-9xks5\" (UID: \"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785\") " pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.145681 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.145610    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wl9d\" (UniqueName: \"kubernetes.io/projected/a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785-kube-api-access-6wl9d\") pod \"model-serving-api-54d89bd5fc-9xks5\" (UID: \"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785\") " pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.246558 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.246525    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785-tls-certs\") pod \"model-serving-api-54d89bd5fc-9xks5\" (UID: \"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785\") " pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.246735 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.246582    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wl9d\" (UniqueName: \"kubernetes.io/projected/a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785-kube-api-access-6wl9d\") pod \"model-serving-api-54d89bd5fc-9xks5\" (UID: \"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785\") " pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.249147 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.249114    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785-tls-certs\") pod \"model-serving-api-54d89bd5fc-9xks5\" (UID: \"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785\") " pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.260353 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.260326    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wl9d\" (UniqueName: \"kubernetes.io/projected/a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785-kube-api-access-6wl9d\") pod \"model-serving-api-54d89bd5fc-9xks5\" (UID: \"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785\") " pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.365268 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.365224    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:24.525095 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:24.525058    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-9xks5"]
Jul 23 17:50:24.529068 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:50:24.529027    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9e4e79b_ddd7_40b3_bdf8_2bebbd11c785.slice/crio-e5b3d512603d8fd0d870f738f87f4963613fb40b93a96db117f4fb8d40ac11aa WatchSource:0}: Error finding container e5b3d512603d8fd0d870f738f87f4963613fb40b93a96db117f4fb8d40ac11aa: Status 404 returned error can't find the container with id e5b3d512603d8fd0d870f738f87f4963613fb40b93a96db117f4fb8d40ac11aa
Jul 23 17:50:25.123958 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:25.123919    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9xks5" event={"ID":"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785","Type":"ContainerStarted","Data":"e5b3d512603d8fd0d870f738f87f4963613fb40b93a96db117f4fb8d40ac11aa"}
Jul 23 17:50:27.134516 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:27.134475    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-9xks5" event={"ID":"a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785","Type":"ContainerStarted","Data":"bba6c8b38e46b031b292141976719d2dbf125ec845c772c56c5f2356423da06e"}
Jul 23 17:50:27.135009 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:27.134652    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:27.136007 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:27.135983    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7" event={"ID":"f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2","Type":"ContainerStarted","Data":"c311ab48cdcbc617e9e42840803dd85e280f152fcbd213bcd0e569422ba144e8"}
Jul 23 17:50:27.167662 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:27.167596    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-9xks5" podStartSLOduration=0.992653029 podStartE2EDuration="3.167574317s" totalImagesPullingTime="2.174921288s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:50:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:50:27.151549091 +0000 UTC m=+321.991057058" watchObservedRunningTime="2026-07-23 17:50:27.167574317 +0000 UTC m=+322.007082265"
Jul 23 17:50:30.260405 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.260341    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5b6c5d7974-k9nj7" podStartSLOduration=5.198033777 podStartE2EDuration="8.260316355s" totalImagesPullingTime="3.062282578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:50:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:50:27.167848214 +0000 UTC m=+322.007356162" watchObservedRunningTime="2026-07-23 17:50:30.260316355 +0000 UTC m=+325.099824304"
Jul 23 17:50:30.261715 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.261685    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"]
Jul 23 17:50:30.264956 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.264942    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.269290 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.269264    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 17:50:30.269290 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.269283    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-qzls6\""
Jul 23 17:50:30.280888 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.280860    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"]
Jul 23 17:50:30.405911 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.405865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fefef8d-63df-4f11-84a0-483681b242cb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.405911 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.405911    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.406118 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.405947    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fefef8d-63df-4f11-84a0-483681b242cb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.406118 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.405994    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.406118 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.406032    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.406118 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.406062    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hflcj\" (UniqueName: \"kubernetes.io/projected/8fefef8d-63df-4f11-84a0-483681b242cb-kube-api-access-hflcj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.406240 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.406120    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.406240 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.406163    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.406240 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.406185    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fefef8d-63df-4f11-84a0-483681b242cb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.507525 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507486    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.507737 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507532    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fefef8d-63df-4f11-84a0-483681b242cb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.507737 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507571    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fefef8d-63df-4f11-84a0-483681b242cb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.507737 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507605    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.507737 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507650    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fefef8d-63df-4f11-84a0-483681b242cb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.507737 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507678    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.507737 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507725    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.508064 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507758    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hflcj\" (UniqueName: \"kubernetes.io/projected/8fefef8d-63df-4f11-84a0-483681b242cb-kube-api-access-hflcj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.508064 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.507809    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.508064 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.508004    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.508213 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.508156    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.508861 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.508432    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.508861 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.508661    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.509026 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.508927    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8fefef8d-63df-4f11-84a0-483681b242cb-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.510483 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.510429    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8fefef8d-63df-4f11-84a0-483681b242cb-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.510831 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.510812    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8fefef8d-63df-4f11-84a0-483681b242cb-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.517371 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.517330    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8fefef8d-63df-4f11-84a0-483681b242cb-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.518571 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.518540    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hflcj\" (UniqueName: \"kubernetes.io/projected/8fefef8d-63df-4f11-84a0-483681b242cb-kube-api-access-hflcj\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4h72j8\" (UID: \"8fefef8d-63df-4f11-84a0-483681b242cb\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.575146 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.575105    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:30.777337 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:30.777257    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"]
Jul 23 17:50:30.781180 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:50:30.781146    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fefef8d_63df_4f11_84a0_483681b242cb.slice/crio-a8c6e6bc2c2f348163f02cdfd9a1a87b437a15605e2ae39abd90e0355b9ac522 WatchSource:0}: Error finding container a8c6e6bc2c2f348163f02cdfd9a1a87b437a15605e2ae39abd90e0355b9ac522: Status 404 returned error can't find the container with id a8c6e6bc2c2f348163f02cdfd9a1a87b437a15605e2ae39abd90e0355b9ac522
Jul 23 17:50:31.152054 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:31.152013    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8" event={"ID":"8fefef8d-63df-4f11-84a0-483681b242cb","Type":"ContainerStarted","Data":"a8c6e6bc2c2f348163f02cdfd9a1a87b437a15605e2ae39abd90e0355b9ac522"}
Jul 23 17:50:32.623714 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.623676    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"]
Jul 23 17:50:32.631429 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.631400    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.636641 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.636608    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 17:50:32.636641 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.636625    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-r55r8\""
Jul 23 17:50:32.636928 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.636608    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 17:50:32.636928 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.636619    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 17:50:32.645252 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.645231    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"]
Jul 23 17:50:32.731615 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.731580    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d01399fd-01d1-45a8-b785-10da9da185cf-manager-config\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.731823 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.731646    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d01399fd-01d1-45a8-b785-10da9da185cf-cert\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.731823 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.731722    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qxrt\" (UniqueName: \"kubernetes.io/projected/d01399fd-01d1-45a8-b785-10da9da185cf-kube-api-access-5qxrt\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.731823 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.731779    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d01399fd-01d1-45a8-b785-10da9da185cf-metrics-cert\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.832331 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.832292    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d01399fd-01d1-45a8-b785-10da9da185cf-manager-config\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.832514 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.832379    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d01399fd-01d1-45a8-b785-10da9da185cf-cert\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.832514 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.832417    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qxrt\" (UniqueName: \"kubernetes.io/projected/d01399fd-01d1-45a8-b785-10da9da185cf-kube-api-access-5qxrt\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.832514 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.832463    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d01399fd-01d1-45a8-b785-10da9da185cf-metrics-cert\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.833120 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.833082    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/d01399fd-01d1-45a8-b785-10da9da185cf-manager-config\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.835189 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.835168    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d01399fd-01d1-45a8-b785-10da9da185cf-cert\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.835368 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.835340    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/d01399fd-01d1-45a8-b785-10da9da185cf-metrics-cert\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.848399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.848369    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qxrt\" (UniqueName: \"kubernetes.io/projected/d01399fd-01d1-45a8-b785-10da9da185cf-kube-api-access-5qxrt\") pod \"lws-controller-manager-754bfdc447-n4bvg\" (UID: \"d01399fd-01d1-45a8-b785-10da9da185cf\") " pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:32.941958 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:32.941862    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:33.383159 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:33.383124    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:50:33.383270 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:33.383208    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:50:33.383270 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:33.383257    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:50:33.448365 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:33.448344    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"]
Jul 23 17:50:33.451547 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:50:33.451522    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd01399fd_01d1_45a8_b785_10da9da185cf.slice/crio-20a43ec6d540bc3c5557e68dffa5e8c4bbdda4ac5370d567b8018399922c275f WatchSource:0}: Error finding container 20a43ec6d540bc3c5557e68dffa5e8c4bbdda4ac5370d567b8018399922c275f: Status 404 returned error can't find the container with id 20a43ec6d540bc3c5557e68dffa5e8c4bbdda4ac5370d567b8018399922c275f
Jul 23 17:50:34.168907 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:34.168862    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg" event={"ID":"d01399fd-01d1-45a8-b785-10da9da185cf","Type":"ContainerStarted","Data":"20a43ec6d540bc3c5557e68dffa5e8c4bbdda4ac5370d567b8018399922c275f"}
Jul 23 17:50:34.170741 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:34.170669    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8" event={"ID":"8fefef8d-63df-4f11-84a0-483681b242cb","Type":"ContainerStarted","Data":"5c311e272ad2e3fe5dc6dd5d9fcabda8823c3b76b1af3ec364e1c7f37c475576"}
Jul 23 17:50:34.216010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:34.215954    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8" podStartSLOduration=1.6167410439999998 podStartE2EDuration="4.21593538s" totalImagesPullingTime="2.599194336s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:50:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:50:34.213168896 +0000 UTC m=+329.052676893" watchObservedRunningTime="2026-07-23 17:50:34.21593538 +0000 UTC m=+329.055443327"
Jul 23 17:50:34.575457 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:34.575412    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:34.581276 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:34.581244    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:35.177866 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:35.177760    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg" event={"ID":"d01399fd-01d1-45a8-b785-10da9da185cf","Type":"ContainerStarted","Data":"2fc77091a77f257f4e5679a3fab1e4f72854c4ac15cefdddb976ffb2ce7c7f65"}
Jul 23 17:50:35.178310 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:35.177982    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:50:35.178310 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:35.178023    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:35.178936 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:35.178918    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4h72j8"
Jul 23 17:50:35.229817 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:35.229766    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg" podStartSLOduration=1.893683182 podStartE2EDuration="3.229747221s" totalImagesPullingTime="1.336064039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:50:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:50:35.227669888 +0000 UTC m=+330.067177835" watchObservedRunningTime="2026-07-23 17:50:35.229747221 +0000 UTC m=+330.069255169"
Jul 23 17:50:38.144655 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:38.144624    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-9xks5"
Jul 23 17:50:46.185139 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:50:46.185107    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-754bfdc447-n4bvg"
Jul 23 17:51:51.527184 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:51:51.527150    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb847698d-lv6t8"]
Jul 23 17:52:12.371194 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.371151    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"]
Jul 23 17:52:12.374253 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.374230    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.377853 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.377832    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7k9z6\""
Jul 23 17:52:12.378017 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.377997    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 17:52:12.378251 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.378235    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 17:52:12.378334 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.378305    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 17:52:12.378472 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.378456    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 17:52:12.387900 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.387874    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"]
Jul 23 17:52:12.507645 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.507604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5da948c6-e962-4e8a-9204-7f9212241836-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.507857 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.507757    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5da948c6-e962-4e8a-9204-7f9212241836-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.507857 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.507783    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfmsn\" (UniqueName: \"kubernetes.io/projected/5da948c6-e962-4e8a-9204-7f9212241836-kube-api-access-pfmsn\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.609175 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.609139    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5da948c6-e962-4e8a-9204-7f9212241836-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.609380 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.609219    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5da948c6-e962-4e8a-9204-7f9212241836-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.609380 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.609239    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfmsn\" (UniqueName: \"kubernetes.io/projected/5da948c6-e962-4e8a-9204-7f9212241836-kube-api-access-pfmsn\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.609380 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:52:12.609318    2582 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 23 17:52:12.609539 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:52:12.609408    2582 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5da948c6-e962-4e8a-9204-7f9212241836-plugin-serving-cert podName:5da948c6-e962-4e8a-9204-7f9212241836 nodeName:}" failed. No retries permitted until 2026-07-23 17:52:13.109384543 +0000 UTC m=+427.948892485 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/5da948c6-e962-4e8a-9204-7f9212241836-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-hcngm" (UID: "5da948c6-e962-4e8a-9204-7f9212241836") : secret "plugin-serving-cert" not found
Jul 23 17:52:12.609952 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.609933    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/5da948c6-e962-4e8a-9204-7f9212241836-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:12.633518 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:12.633444    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfmsn\" (UniqueName: \"kubernetes.io/projected/5da948c6-e962-4e8a-9204-7f9212241836-kube-api-access-pfmsn\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:13.113962 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:13.113921    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5da948c6-e962-4e8a-9204-7f9212241836-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:13.116324 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:13.116303    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/5da948c6-e962-4e8a-9204-7f9212241836-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-hcngm\" (UID: \"5da948c6-e962-4e8a-9204-7f9212241836\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:13.283510 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:13.283467    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"
Jul 23 17:52:13.450973 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:13.450939    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm"]
Jul 23 17:52:13.454288 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:52:13.454256    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5da948c6_e962_4e8a_9204_7f9212241836.slice/crio-11a834293cb8b566e1ebbf1c2e361aebfa1c8e42ea685e7dbc7f8c89f63203fb WatchSource:0}: Error finding container 11a834293cb8b566e1ebbf1c2e361aebfa1c8e42ea685e7dbc7f8c89f63203fb: Status 404 returned error can't find the container with id 11a834293cb8b566e1ebbf1c2e361aebfa1c8e42ea685e7dbc7f8c89f63203fb
Jul 23 17:52:13.588082 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:13.588041    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm" event={"ID":"5da948c6-e962-4e8a-9204-7f9212241836","Type":"ContainerStarted","Data":"11a834293cb8b566e1ebbf1c2e361aebfa1c8e42ea685e7dbc7f8c89f63203fb"}
Jul 23 17:52:16.546945 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.546905    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bb847698d-lv6t8" podUID="fd1eb2cb-2f79-44cf-9697-c3ae68288501" containerName="console" containerID="cri-o://01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51" gracePeriod=15
Jul 23 17:52:16.831235 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.831210    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:52:16.948479 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.948442    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcv4v\" (UniqueName: \"kubernetes.io/projected/fd1eb2cb-2f79-44cf-9697-c3ae68288501-kube-api-access-bcv4v\") pod \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") "
Jul 23 17:52:16.948479 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.948497    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-config\") pod \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") "
Jul 23 17:52:16.948782 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.948564    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-trusted-ca-bundle\") pod \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") "
Jul 23 17:52:16.948782 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.948605    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-oauth-config\") pod \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") "
Jul 23 17:52:16.948782 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.948640    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-serving-cert\") pod \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") "
Jul 23 17:52:16.948782 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.948679    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-oauth-serving-cert\") pod \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") "
Jul 23 17:52:16.948782 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.948739    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-service-ca\") pod \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\" (UID: \"fd1eb2cb-2f79-44cf-9697-c3ae68288501\") "
Jul 23 17:52:16.949367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.949293    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-config" (OuterVolumeSpecName: "console-config") pod "fd1eb2cb-2f79-44cf-9697-c3ae68288501" (UID: "fd1eb2cb-2f79-44cf-9697-c3ae68288501"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:52:16.949367 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.949260    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd1eb2cb-2f79-44cf-9697-c3ae68288501" (UID: "fd1eb2cb-2f79-44cf-9697-c3ae68288501"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:52:16.949553 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.949415    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd1eb2cb-2f79-44cf-9697-c3ae68288501" (UID: "fd1eb2cb-2f79-44cf-9697-c3ae68288501"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:52:16.949553 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.949431    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd1eb2cb-2f79-44cf-9697-c3ae68288501" (UID: "fd1eb2cb-2f79-44cf-9697-c3ae68288501"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 17:52:16.951586 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.951558    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd1eb2cb-2f79-44cf-9697-c3ae68288501" (UID: "fd1eb2cb-2f79-44cf-9697-c3ae68288501"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:52:16.951978 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.951951    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd1eb2cb-2f79-44cf-9697-c3ae68288501" (UID: "fd1eb2cb-2f79-44cf-9697-c3ae68288501"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:52:16.951978 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:16.951959    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd1eb2cb-2f79-44cf-9697-c3ae68288501-kube-api-access-bcv4v" (OuterVolumeSpecName: "kube-api-access-bcv4v") pod "fd1eb2cb-2f79-44cf-9697-c3ae68288501" (UID: "fd1eb2cb-2f79-44cf-9697-c3ae68288501"). InnerVolumeSpecName "kube-api-access-bcv4v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:52:17.049454 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.049422    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcv4v\" (UniqueName: \"kubernetes.io/projected/fd1eb2cb-2f79-44cf-9697-c3ae68288501-kube-api-access-bcv4v\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:52:17.049454 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.049451    2582 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:52:17.049735 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.049466    2582 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-trusted-ca-bundle\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:52:17.049735 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.049481    2582 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-oauth-config\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:52:17.049735 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.049494    2582 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd1eb2cb-2f79-44cf-9697-c3ae68288501-console-serving-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:52:17.049735 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.049506    2582 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-oauth-serving-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:52:17.049735 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.049519    2582 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd1eb2cb-2f79-44cf-9697-c3ae68288501-service-ca\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:52:17.607768 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.607698    2582 generic.go:356] "Generic (PLEG): container finished" podID="fd1eb2cb-2f79-44cf-9697-c3ae68288501" containerID="01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51" exitCode=0
Jul 23 17:52:17.608296 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.607775    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bb847698d-lv6t8"
Jul 23 17:52:17.608296 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.607780    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb847698d-lv6t8" event={"ID":"fd1eb2cb-2f79-44cf-9697-c3ae68288501","Type":"ContainerDied","Data":"01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51"}
Jul 23 17:52:17.608296 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.607905    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bb847698d-lv6t8" event={"ID":"fd1eb2cb-2f79-44cf-9697-c3ae68288501","Type":"ContainerDied","Data":"a2bb8892b806681e7ee681678f71bb421e508b1c0d4b0bf5012b238b981f87c9"}
Jul 23 17:52:17.608296 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.607948    2582 scope.go:117] "RemoveContainer" containerID="01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51"
Jul 23 17:52:17.619574 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.619550    2582 scope.go:117] "RemoveContainer" containerID="01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51"
Jul 23 17:52:17.619927 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:52:17.619901    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51\": container with ID starting with 01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51 not found: ID does not exist" containerID="01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51"
Jul 23 17:52:17.620015 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.619937    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51"} err="failed to get container status \"01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51\": rpc error: code = NotFound desc = could not find container \"01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51\": container with ID starting with 01208053e216d016e566b56d5ce316156b3528a0cf0d35c57e95a1e20d044d51 not found: ID does not exist"
Jul 23 17:52:17.645343 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.645300    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bb847698d-lv6t8"]
Jul 23 17:52:17.657776 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.657696    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bb847698d-lv6t8"]
Jul 23 17:52:17.784848 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:17.784814    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd1eb2cb-2f79-44cf-9697-c3ae68288501" path="/var/lib/kubelet/pods/fd1eb2cb-2f79-44cf-9697-c3ae68288501/volumes"
Jul 23 17:52:39.720375 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:39.720334    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm" event={"ID":"5da948c6-e962-4e8a-9204-7f9212241836","Type":"ContainerStarted","Data":"a20864923a1fd299b5929a416406f49f5bacf5ca42d6904a09258bad4bf62723"}
Jul 23 17:52:39.738418 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:39.738360    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-hcngm" podStartSLOduration=2.568613023 podStartE2EDuration="27.73834319s" totalImagesPullingTime="25.169730167s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:52:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:52:39.736586 +0000 UTC m=+454.576093948" watchObservedRunningTime="2026-07-23 17:52:39.73834319 +0000 UTC m=+454.577851137"
Jul 23 17:52:57.172579 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.172545    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"]
Jul 23 17:52:57.173059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.172870    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd1eb2cb-2f79-44cf-9697-c3ae68288501" containerName="console"
Jul 23 17:52:57.173059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.172881    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd1eb2cb-2f79-44cf-9697-c3ae68288501" containerName="console"
Jul 23 17:52:57.173059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.172928    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd1eb2cb-2f79-44cf-9697-c3ae68288501" containerName="console"
Jul 23 17:52:57.236793 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.236758    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"]
Jul 23 17:52:57.236979 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.236861    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.239687 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.239667    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-ddhk5\""
Jul 23 17:52:57.319013 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.318974    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319013 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319021    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319349 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319043    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319349 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319092    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319349 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319200    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319349 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319227    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319349 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319254    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319349 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319342    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g88zm\" (UniqueName: \"kubernetes.io/projected/d2eb6cca-0902-4227-8a43-b223e53fd3b2-kube-api-access-g88zm\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.319541 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.319389    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.420752 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.420691    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.420935 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.420778    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.420935 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.420817    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.420935 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.420837    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421086 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421056    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421166 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421145    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421242 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421149    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421242 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421192    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421242 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421200    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421398 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421251    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421398 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421325    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421398 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g88zm\" (UniqueName: \"kubernetes.io/projected/d2eb6cca-0902-4227-8a43-b223e53fd3b2-kube-api-access-g88zm\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421419    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.421853 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.421828    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.423240 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.423187    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.423699 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.423682    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.432571 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.432541    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g88zm\" (UniqueName: \"kubernetes.io/projected/d2eb6cca-0902-4227-8a43-b223e53fd3b2-kube-api-access-g88zm\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.432801 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.432782    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d2eb6cca-0902-4227-8a43-b223e53fd3b2-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-m5sn9\" (UID: \"d2eb6cca-0902-4227-8a43-b223e53fd3b2\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.547592 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.547555    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:57.691420 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.691393    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"]
Jul 23 17:52:57.693511 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:52:57.693480    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2eb6cca_0902_4227_8a43_b223e53fd3b2.slice/crio-72d8d2dd0190789eab1c72dc1a4cdf48d86e6836d188e35a94a06a990a5fde5a WatchSource:0}: Error finding container 72d8d2dd0190789eab1c72dc1a4cdf48d86e6836d188e35a94a06a990a5fde5a: Status 404 returned error can't find the container with id 72d8d2dd0190789eab1c72dc1a4cdf48d86e6836d188e35a94a06a990a5fde5a
Jul 23 17:52:57.695610 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.695583    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:52:57.695691 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.695642    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:52:57.695691 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.695670    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:52:57.802653 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.802617    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" event={"ID":"d2eb6cca-0902-4227-8a43-b223e53fd3b2","Type":"ContainerStarted","Data":"14213f1c6d946517e47c3d740d7b20b3177e98ca92b3479276f4a206c4c98129"}
Jul 23 17:52:57.802849 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.802679    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" event={"ID":"d2eb6cca-0902-4227-8a43-b223e53fd3b2","Type":"ContainerStarted","Data":"72d8d2dd0190789eab1c72dc1a4cdf48d86e6836d188e35a94a06a990a5fde5a"}
Jul 23 17:52:57.827641 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:57.827571    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" podStartSLOduration=0.827550318 podStartE2EDuration="827.550318ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:52:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:52:57.825022559 +0000 UTC m=+472.664530508" watchObservedRunningTime="2026-07-23 17:52:57.827550318 +0000 UTC m=+472.667058267"
Jul 23 17:52:58.548024 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:58.547978    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:58.552864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:58.552842    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:58.807540 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:58.807512    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:58.808524 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:58.808505    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:52:59.273381 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.273300    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:52:59.275401 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.275385    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.278922 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.278899    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 23 17:52:59.291274 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.291248    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:52:59.353123 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.350768    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:52:59.439614 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.439569    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-887v5\" (UniqueName: \"kubernetes.io/projected/02174c34-a9c8-45b4-829f-c79cc0adfdd3-kube-api-access-887v5\") pod \"limitador-limitador-54d6d988d5-x5svp\" (UID: \"02174c34-a9c8-45b4-829f-c79cc0adfdd3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.439614 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.439618    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02174c34-a9c8-45b4-829f-c79cc0adfdd3-config-file\") pod \"limitador-limitador-54d6d988d5-x5svp\" (UID: \"02174c34-a9c8-45b4-829f-c79cc0adfdd3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.540261 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.540162    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-887v5\" (UniqueName: \"kubernetes.io/projected/02174c34-a9c8-45b4-829f-c79cc0adfdd3-kube-api-access-887v5\") pod \"limitador-limitador-54d6d988d5-x5svp\" (UID: \"02174c34-a9c8-45b4-829f-c79cc0adfdd3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.540261 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.540212    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02174c34-a9c8-45b4-829f-c79cc0adfdd3-config-file\") pod \"limitador-limitador-54d6d988d5-x5svp\" (UID: \"02174c34-a9c8-45b4-829f-c79cc0adfdd3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.540904 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.540886    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/02174c34-a9c8-45b4-829f-c79cc0adfdd3-config-file\") pod \"limitador-limitador-54d6d988d5-x5svp\" (UID: \"02174c34-a9c8-45b4-829f-c79cc0adfdd3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.549762 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.549739    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-887v5\" (UniqueName: \"kubernetes.io/projected/02174c34-a9c8-45b4-829f-c79cc0adfdd3-kube-api-access-887v5\") pod \"limitador-limitador-54d6d988d5-x5svp\" (UID: \"02174c34-a9c8-45b4-829f-c79cc0adfdd3\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.585895 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.585849    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:52:59.729486 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.729451    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:52:59.732718 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:52:59.732675    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02174c34_a9c8_45b4_829f_c79cc0adfdd3.slice/crio-e656de5fe9e998fbaa80049c1c188aa461f1e229e74e534f73366c44573e125d WatchSource:0}: Error finding container e656de5fe9e998fbaa80049c1c188aa461f1e229e74e534f73366c44573e125d: Status 404 returned error can't find the container with id e656de5fe9e998fbaa80049c1c188aa461f1e229e74e534f73366c44573e125d
Jul 23 17:52:59.812263 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.812226    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp" event={"ID":"02174c34-a9c8-45b4-829f-c79cc0adfdd3","Type":"ContainerStarted","Data":"e656de5fe9e998fbaa80049c1c188aa461f1e229e74e534f73366c44573e125d"}
Jul 23 17:52:59.952378 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.952348    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-7qm2c"]
Jul 23 17:52:59.955054 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.955029    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qm2c"
Jul 23 17:52:59.958656 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.958637    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fz58f\""
Jul 23 17:52:59.965907 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:52:59.965880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qm2c"]
Jul 23 17:53:00.046075 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:00.046038    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj2sh\" (UniqueName: \"kubernetes.io/projected/e98aa646-48c9-4d3b-abf5-4c53a602f848-kube-api-access-vj2sh\") pod \"authorino-58cd955675-7qm2c\" (UID: \"e98aa646-48c9-4d3b-abf5-4c53a602f848\") " pod="kuadrant-system/authorino-58cd955675-7qm2c"
Jul 23 17:53:00.146837 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:00.146742    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj2sh\" (UniqueName: \"kubernetes.io/projected/e98aa646-48c9-4d3b-abf5-4c53a602f848-kube-api-access-vj2sh\") pod \"authorino-58cd955675-7qm2c\" (UID: \"e98aa646-48c9-4d3b-abf5-4c53a602f848\") " pod="kuadrant-system/authorino-58cd955675-7qm2c"
Jul 23 17:53:00.157131 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:00.157108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj2sh\" (UniqueName: \"kubernetes.io/projected/e98aa646-48c9-4d3b-abf5-4c53a602f848-kube-api-access-vj2sh\") pod \"authorino-58cd955675-7qm2c\" (UID: \"e98aa646-48c9-4d3b-abf5-4c53a602f848\") " pod="kuadrant-system/authorino-58cd955675-7qm2c"
Jul 23 17:53:00.263915 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:00.263876    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qm2c"
Jul 23 17:53:00.404229 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:00.404132    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qm2c"]
Jul 23 17:53:00.407848 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:53:00.407812    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode98aa646_48c9_4d3b_abf5_4c53a602f848.slice/crio-683cb8dc9d930ea4c2c4a32c71635fb1b300b8dc02be4798d1240698c46f6c7b WatchSource:0}: Error finding container 683cb8dc9d930ea4c2c4a32c71635fb1b300b8dc02be4798d1240698c46f6c7b: Status 404 returned error can't find the container with id 683cb8dc9d930ea4c2c4a32c71635fb1b300b8dc02be4798d1240698c46f6c7b
Jul 23 17:53:00.820024 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:00.819981    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qm2c" event={"ID":"e98aa646-48c9-4d3b-abf5-4c53a602f848","Type":"ContainerStarted","Data":"683cb8dc9d930ea4c2c4a32c71635fb1b300b8dc02be4798d1240698c46f6c7b"}
Jul 23 17:53:04.707823 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:04.707789    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qm2c"]
Jul 23 17:53:04.847134 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:04.847092    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qm2c" event={"ID":"e98aa646-48c9-4d3b-abf5-4c53a602f848","Type":"ContainerStarted","Data":"2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6"}
Jul 23 17:53:04.847134 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:04.847107    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-7qm2c" podUID="e98aa646-48c9-4d3b-abf5-4c53a602f848" containerName="authorino" containerID="cri-o://2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6" gracePeriod=30
Jul 23 17:53:04.849083 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:04.849037    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp" event={"ID":"02174c34-a9c8-45b4-829f-c79cc0adfdd3","Type":"ContainerStarted","Data":"7ecbb31a89fee994bd26184e33d19d1436f1b2d9c9ca37dde324f7f600143913"}
Jul 23 17:53:04.849210 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:04.849190    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:53:04.865012 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:04.864954    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-7qm2c" podStartSLOduration=1.594508669 podStartE2EDuration="5.864933784s" totalImagesPullingTime="4.270425115s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:53:04.863619199 +0000 UTC m=+479.703127144" watchObservedRunningTime="2026-07-23 17:53:04.864933784 +0000 UTC m=+479.704441735"
Jul 23 17:53:04.888357 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:04.888300    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp" podStartSLOduration=1.086606685 podStartE2EDuration="5.88828372s" totalImagesPullingTime="4.801677035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:52:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:53:04.887001072 +0000 UTC m=+479.726509021" watchObservedRunningTime="2026-07-23 17:53:04.88828372 +0000 UTC m=+479.727791668"
Jul 23 17:53:05.098902 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.098835    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qm2c"
Jul 23 17:53:05.196796 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.196767    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vj2sh\" (UniqueName: \"kubernetes.io/projected/e98aa646-48c9-4d3b-abf5-4c53a602f848-kube-api-access-vj2sh\") pod \"e98aa646-48c9-4d3b-abf5-4c53a602f848\" (UID: \"e98aa646-48c9-4d3b-abf5-4c53a602f848\") "
Jul 23 17:53:05.198954 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.198928    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e98aa646-48c9-4d3b-abf5-4c53a602f848-kube-api-access-vj2sh" (OuterVolumeSpecName: "kube-api-access-vj2sh") pod "e98aa646-48c9-4d3b-abf5-4c53a602f848" (UID: "e98aa646-48c9-4d3b-abf5-4c53a602f848"). InnerVolumeSpecName "kube-api-access-vj2sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:53:05.302480 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.297949    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vj2sh\" (UniqueName: \"kubernetes.io/projected/e98aa646-48c9-4d3b-abf5-4c53a602f848-kube-api-access-vj2sh\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:53:05.854682 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.854588    2582 generic.go:356] "Generic (PLEG): container finished" podID="e98aa646-48c9-4d3b-abf5-4c53a602f848" containerID="2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6" exitCode=0
Jul 23 17:53:05.854682 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.854639    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-7qm2c"
Jul 23 17:53:05.854682 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.854669    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qm2c" event={"ID":"e98aa646-48c9-4d3b-abf5-4c53a602f848","Type":"ContainerDied","Data":"2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6"}
Jul 23 17:53:05.855213 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.854723    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-7qm2c" event={"ID":"e98aa646-48c9-4d3b-abf5-4c53a602f848","Type":"ContainerDied","Data":"683cb8dc9d930ea4c2c4a32c71635fb1b300b8dc02be4798d1240698c46f6c7b"}
Jul 23 17:53:05.855213 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.854743    2582 scope.go:117] "RemoveContainer" containerID="2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6"
Jul 23 17:53:05.862696 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.862677    2582 scope.go:117] "RemoveContainer" containerID="2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6"
Jul 23 17:53:05.863021 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:53:05.862995    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6\": container with ID starting with 2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6 not found: ID does not exist" containerID="2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6"
Jul 23 17:53:05.863128 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.863029    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6"} err="failed to get container status \"2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6\": rpc error: code = NotFound desc = could not find container \"2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6\": container with ID starting with 2ebfb80a0d4536c78a72f192f906bc65fa4a6c9b43ce202d2ccf16b0941b18d6 not found: ID does not exist"
Jul 23 17:53:05.879530 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.879498    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qm2c"]
Jul 23 17:53:05.896187 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:05.896157    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-7qm2c"]
Jul 23 17:53:07.784212 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:07.784168    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e98aa646-48c9-4d3b-abf5-4c53a602f848" path="/var/lib/kubelet/pods/e98aa646-48c9-4d3b-abf5-4c53a602f848/volumes"
Jul 23 17:53:15.855453 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:15.855423    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-x5svp"
Jul 23 17:53:33.458341 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.458304    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g9jg7"]
Jul 23 17:53:33.458746 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.458644    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e98aa646-48c9-4d3b-abf5-4c53a602f848" containerName="authorino"
Jul 23 17:53:33.458746 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.458657    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="e98aa646-48c9-4d3b-abf5-4c53a602f848" containerName="authorino"
Jul 23 17:53:33.458825 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.458747    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="e98aa646-48c9-4d3b-abf5-4c53a602f848" containerName="authorino"
Jul 23 17:53:33.463611 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.463588    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g9jg7"
Jul 23 17:53:33.467392 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.467368    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-fz58f\""
Jul 23 17:53:33.489175 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.489138    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g9jg7"]
Jul 23 17:53:33.554116 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.554075    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6wl8\" (UniqueName: \"kubernetes.io/projected/7dd4b0d2-4566-4b30-ab93-9971828fe048-kube-api-access-z6wl8\") pod \"authorino-5d4b67c446-g9jg7\" (UID: \"7dd4b0d2-4566-4b30-ab93-9971828fe048\") " pod="kuadrant-system/authorino-5d4b67c446-g9jg7"
Jul 23 17:53:33.655094 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.655059    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6wl8\" (UniqueName: \"kubernetes.io/projected/7dd4b0d2-4566-4b30-ab93-9971828fe048-kube-api-access-z6wl8\") pod \"authorino-5d4b67c446-g9jg7\" (UID: \"7dd4b0d2-4566-4b30-ab93-9971828fe048\") " pod="kuadrant-system/authorino-5d4b67c446-g9jg7"
Jul 23 17:53:33.666774 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.666748    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6wl8\" (UniqueName: \"kubernetes.io/projected/7dd4b0d2-4566-4b30-ab93-9971828fe048-kube-api-access-z6wl8\") pod \"authorino-5d4b67c446-g9jg7\" (UID: \"7dd4b0d2-4566-4b30-ab93-9971828fe048\") " pod="kuadrant-system/authorino-5d4b67c446-g9jg7"
Jul 23 17:53:33.728672 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.728590    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g9jg7"]
Jul 23 17:53:33.728984 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.728967    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g9jg7"
Jul 23 17:53:33.760559 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.760522    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-57c5b5f5d7-c8vzw"]
Jul 23 17:53:33.765649 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.765625    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw"
Jul 23 17:53:33.777008 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.776979    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-57c5b5f5d7-c8vzw"]
Jul 23 17:53:33.856972 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.856936    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkqbb\" (UniqueName: \"kubernetes.io/projected/823d198c-023a-47c7-8ebe-32af71d74156-kube-api-access-vkqbb\") pod \"authorino-57c5b5f5d7-c8vzw\" (UID: \"823d198c-023a-47c7-8ebe-32af71d74156\") " pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw"
Jul 23 17:53:33.878012 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.877976    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g9jg7"]
Jul 23 17:53:33.881985 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:53:33.881948    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7dd4b0d2_4566_4b30_ab93_9971828fe048.slice/crio-7dd9e3774646c9ad5009be299775351f7a6b17a5a63268a4e376946cf3faa38c WatchSource:0}: Error finding container 7dd9e3774646c9ad5009be299775351f7a6b17a5a63268a4e376946cf3faa38c: Status 404 returned error can't find the container with id 7dd9e3774646c9ad5009be299775351f7a6b17a5a63268a4e376946cf3faa38c
Jul 23 17:53:33.958332 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.958294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkqbb\" (UniqueName: \"kubernetes.io/projected/823d198c-023a-47c7-8ebe-32af71d74156-kube-api-access-vkqbb\") pod \"authorino-57c5b5f5d7-c8vzw\" (UID: \"823d198c-023a-47c7-8ebe-32af71d74156\") " pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw"
Jul 23 17:53:33.970629 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.970596    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkqbb\" (UniqueName: \"kubernetes.io/projected/823d198c-023a-47c7-8ebe-32af71d74156-kube-api-access-vkqbb\") pod \"authorino-57c5b5f5d7-c8vzw\" (UID: \"823d198c-023a-47c7-8ebe-32af71d74156\") " pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw"
Jul 23 17:53:33.972260 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:33.972235    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g9jg7" event={"ID":"7dd4b0d2-4566-4b30-ab93-9971828fe048","Type":"ContainerStarted","Data":"7dd9e3774646c9ad5009be299775351f7a6b17a5a63268a4e376946cf3faa38c"}
Jul 23 17:53:34.058382 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.058340    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57c5b5f5d7-c8vzw"]
Jul 23 17:53:34.058657 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.058640    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw"
Jul 23 17:53:34.091537 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.091504    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f79ff6667-cqsjf"]
Jul 23 17:53:34.095972 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.095944    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.098940 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.098911    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 23 17:53:34.106971 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.106880    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f79ff6667-cqsjf"]
Jul 23 17:53:34.159241 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.159205    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23f67b90-a0ca-4700-b09c-e9155edadfc3-tls-cert\") pod \"authorino-f79ff6667-cqsjf\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") " pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.159425 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.159274    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9gbw\" (UniqueName: \"kubernetes.io/projected/23f67b90-a0ca-4700-b09c-e9155edadfc3-kube-api-access-q9gbw\") pod \"authorino-f79ff6667-cqsjf\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") " pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.195720 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.195672    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57c5b5f5d7-c8vzw"]
Jul 23 17:53:34.199261 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:53:34.199234    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod823d198c_023a_47c7_8ebe_32af71d74156.slice/crio-af7e8bf1c49c18178375ff9cc71b4c03b6f6a9aeb5afa96f83b6615deff76f74 WatchSource:0}: Error finding container af7e8bf1c49c18178375ff9cc71b4c03b6f6a9aeb5afa96f83b6615deff76f74: Status 404 returned error can't find the container with id af7e8bf1c49c18178375ff9cc71b4c03b6f6a9aeb5afa96f83b6615deff76f74
Jul 23 17:53:34.259928 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.259898    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9gbw\" (UniqueName: \"kubernetes.io/projected/23f67b90-a0ca-4700-b09c-e9155edadfc3-kube-api-access-q9gbw\") pod \"authorino-f79ff6667-cqsjf\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") " pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.260094 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.260076    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23f67b90-a0ca-4700-b09c-e9155edadfc3-tls-cert\") pod \"authorino-f79ff6667-cqsjf\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") " pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.262376 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.262358    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23f67b90-a0ca-4700-b09c-e9155edadfc3-tls-cert\") pod \"authorino-f79ff6667-cqsjf\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") " pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.269951 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.269924    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9gbw\" (UniqueName: \"kubernetes.io/projected/23f67b90-a0ca-4700-b09c-e9155edadfc3-kube-api-access-q9gbw\") pod \"authorino-f79ff6667-cqsjf\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") " pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.407344 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.407252    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:53:34.544315 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.544286    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f79ff6667-cqsjf"]
Jul 23 17:53:34.547094 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:53:34.547067    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod23f67b90_a0ca_4700_b09c_e9155edadfc3.slice/crio-3218f3a2502cc76b4f1b3c95f56dc93ed2860760c2b55c5cd976de5336da22c0 WatchSource:0}: Error finding container 3218f3a2502cc76b4f1b3c95f56dc93ed2860760c2b55c5cd976de5336da22c0: Status 404 returned error can't find the container with id 3218f3a2502cc76b4f1b3c95f56dc93ed2860760c2b55c5cd976de5336da22c0
Jul 23 17:53:34.978272 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.978239    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw" event={"ID":"823d198c-023a-47c7-8ebe-32af71d74156","Type":"ContainerStarted","Data":"3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28"}
Jul 23 17:53:34.978470 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.978286    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw" event={"ID":"823d198c-023a-47c7-8ebe-32af71d74156","Type":"ContainerStarted","Data":"af7e8bf1c49c18178375ff9cc71b4c03b6f6a9aeb5afa96f83b6615deff76f74"}
Jul 23 17:53:34.978470 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.978277    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw" podUID="823d198c-023a-47c7-8ebe-32af71d74156" containerName="authorino" containerID="cri-o://3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28" gracePeriod=30
Jul 23 17:53:34.980032 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.980000    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g9jg7" event={"ID":"7dd4b0d2-4566-4b30-ab93-9971828fe048","Type":"ContainerStarted","Data":"47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b"}
Jul 23 17:53:34.980147 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.980049    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-g9jg7" podUID="7dd4b0d2-4566-4b30-ab93-9971828fe048" containerName="authorino" containerID="cri-o://47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b" gracePeriod=30
Jul 23 17:53:34.981357 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.981334    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f79ff6667-cqsjf" event={"ID":"23f67b90-a0ca-4700-b09c-e9155edadfc3","Type":"ContainerStarted","Data":"3218f3a2502cc76b4f1b3c95f56dc93ed2860760c2b55c5cd976de5336da22c0"}
Jul 23 17:53:34.996968 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:34.996911    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw" podStartSLOduration=1.655653881 podStartE2EDuration="1.99688734s" totalImagesPullingTime="341.233459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:53:34.995562156 +0000 UTC m=+509.835070107" watchObservedRunningTime="2026-07-23 17:53:34.99688734 +0000 UTC m=+509.836395295"
Jul 23 17:53:35.018819 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.018770    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-g9jg7" podStartSLOduration=1.666765144 podStartE2EDuration="2.018750087s" totalImagesPullingTime="351.984943ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:53:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:53:35.017392609 +0000 UTC m=+509.856900557" watchObservedRunningTime="2026-07-23 17:53:35.018750087 +0000 UTC m=+509.858258036"
Jul 23 17:53:35.225250 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.225224    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g9jg7"
Jul 23 17:53:35.256338 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.256311    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw"
Jul 23 17:53:35.370829 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.370793    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vkqbb\" (UniqueName: \"kubernetes.io/projected/823d198c-023a-47c7-8ebe-32af71d74156-kube-api-access-vkqbb\") pod \"823d198c-023a-47c7-8ebe-32af71d74156\" (UID: \"823d198c-023a-47c7-8ebe-32af71d74156\") "
Jul 23 17:53:35.371010 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.370855    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z6wl8\" (UniqueName: \"kubernetes.io/projected/7dd4b0d2-4566-4b30-ab93-9971828fe048-kube-api-access-z6wl8\") pod \"7dd4b0d2-4566-4b30-ab93-9971828fe048\" (UID: \"7dd4b0d2-4566-4b30-ab93-9971828fe048\") "
Jul 23 17:53:35.372963 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.372934    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/823d198c-023a-47c7-8ebe-32af71d74156-kube-api-access-vkqbb" (OuterVolumeSpecName: "kube-api-access-vkqbb") pod "823d198c-023a-47c7-8ebe-32af71d74156" (UID: "823d198c-023a-47c7-8ebe-32af71d74156"). InnerVolumeSpecName "kube-api-access-vkqbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:53:35.372963 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.372949    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dd4b0d2-4566-4b30-ab93-9971828fe048-kube-api-access-z6wl8" (OuterVolumeSpecName: "kube-api-access-z6wl8") pod "7dd4b0d2-4566-4b30-ab93-9971828fe048" (UID: "7dd4b0d2-4566-4b30-ab93-9971828fe048"). InnerVolumeSpecName "kube-api-access-z6wl8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:53:35.471986 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.471902    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vkqbb\" (UniqueName: \"kubernetes.io/projected/823d198c-023a-47c7-8ebe-32af71d74156-kube-api-access-vkqbb\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:53:35.471986 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.471932    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z6wl8\" (UniqueName: \"kubernetes.io/projected/7dd4b0d2-4566-4b30-ab93-9971828fe048-kube-api-access-z6wl8\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:53:35.987017 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.986978    2582 generic.go:356] "Generic (PLEG): container finished" podID="7dd4b0d2-4566-4b30-ab93-9971828fe048" containerID="47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b" exitCode=0
Jul 23 17:53:35.987017 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.987008    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g9jg7" event={"ID":"7dd4b0d2-4566-4b30-ab93-9971828fe048","Type":"ContainerDied","Data":"47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b"}
Jul 23 17:53:35.987558 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.987055    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-g9jg7" event={"ID":"7dd4b0d2-4566-4b30-ab93-9971828fe048","Type":"ContainerDied","Data":"7dd9e3774646c9ad5009be299775351f7a6b17a5a63268a4e376946cf3faa38c"}
Jul 23 17:53:35.987558 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.987073    2582 scope.go:117] "RemoveContainer" containerID="47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b"
Jul 23 17:53:35.987558 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.987022    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-g9jg7"
Jul 23 17:53:35.988960 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.988932    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f79ff6667-cqsjf" event={"ID":"23f67b90-a0ca-4700-b09c-e9155edadfc3","Type":"ContainerStarted","Data":"f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29"}
Jul 23 17:53:35.990328 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.990309    2582 generic.go:356] "Generic (PLEG): container finished" podID="823d198c-023a-47c7-8ebe-32af71d74156" containerID="3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28" exitCode=0
Jul 23 17:53:35.990420 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.990339    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw" event={"ID":"823d198c-023a-47c7-8ebe-32af71d74156","Type":"ContainerDied","Data":"3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28"}
Jul 23 17:53:35.990420 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.990350    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw"
Jul 23 17:53:35.990420 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.990359    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-57c5b5f5d7-c8vzw" event={"ID":"823d198c-023a-47c7-8ebe-32af71d74156","Type":"ContainerDied","Data":"af7e8bf1c49c18178375ff9cc71b4c03b6f6a9aeb5afa96f83b6615deff76f74"}
Jul 23 17:53:35.996088 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.996070    2582 scope.go:117] "RemoveContainer" containerID="47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b"
Jul 23 17:53:35.996352 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:53:35.996335    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b\": container with ID starting with 47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b not found: ID does not exist" containerID="47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b"
Jul 23 17:53:35.996399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.996362    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b"} err="failed to get container status \"47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b\": rpc error: code = NotFound desc = could not find container \"47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b\": container with ID starting with 47d76d66a243610df725c1523c10142beced3b9533fe192ffdd653d9b0149d5b not found: ID does not exist"
Jul 23 17:53:35.996399 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:35.996379    2582 scope.go:117] "RemoveContainer" containerID="3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28"
Jul 23 17:53:36.003612 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:36.003593    2582 scope.go:117] "RemoveContainer" containerID="3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28"
Jul 23 17:53:36.003941 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:53:36.003922    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28\": container with ID starting with 3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28 not found: ID does not exist" containerID="3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28"
Jul 23 17:53:36.003995 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:36.003948    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28"} err="failed to get container status \"3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28\": rpc error: code = NotFound desc = could not find container \"3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28\": container with ID starting with 3bb22ad5a607fd440423729c1b230324e0a83741c8ea99aff3f505dea225de28 not found: ID does not exist"
Jul 23 17:53:36.007406 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:36.007384    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g9jg7"]
Jul 23 17:53:36.012988 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:36.012967    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-g9jg7"]
Jul 23 17:53:36.030805 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:36.030756    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-f79ff6667-cqsjf" podStartSLOduration=1.64201824 podStartE2EDuration="2.030739707s" totalImagesPullingTime="388.721467ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:53:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:53:36.029342499 +0000 UTC m=+510.868850448" watchObservedRunningTime="2026-07-23 17:53:36.030739707 +0000 UTC m=+510.870247652"
Jul 23 17:53:36.046572 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:36.046471    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-57c5b5f5d7-c8vzw"]
Jul 23 17:53:36.051551 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:36.051524    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-57c5b5f5d7-c8vzw"]
Jul 23 17:53:37.788940 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:37.786518    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dd4b0d2-4566-4b30-ab93-9971828fe048" path="/var/lib/kubelet/pods/7dd4b0d2-4566-4b30-ab93-9971828fe048/volumes"
Jul 23 17:53:37.788940 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:53:37.787054    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="823d198c-023a-47c7-8ebe-32af71d74156" path="/var/lib/kubelet/pods/823d198c-023a-47c7-8ebe-32af71d74156/volumes"
Jul 23 17:54:18.247039 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:18.247004    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:54:26.213233 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:26.213199    2582 generic.go:356] "Generic (PLEG): container finished" podID="d2eb6cca-0902-4227-8a43-b223e53fd3b2" containerID="14213f1c6d946517e47c3d740d7b20b3177e98ca92b3479276f4a206c4c98129" exitCode=0
Jul 23 17:54:26.213647 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:26.213272    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" event={"ID":"d2eb6cca-0902-4227-8a43-b223e53fd3b2","Type":"ContainerDied","Data":"14213f1c6d946517e47c3d740d7b20b3177e98ca92b3479276f4a206c4c98129"}
Jul 23 17:54:26.213691 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:26.213664    2582 scope.go:117] "RemoveContainer" containerID="14213f1c6d946517e47c3d740d7b20b3177e98ca92b3479276f4a206c4c98129"
Jul 23 17:54:26.214732 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:26.214688    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:54:26.214789 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:26.214768    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:54:26.214822 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:26.214804    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 17:54:27.220586 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:27.220553    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" event={"ID":"d2eb6cca-0902-4227-8a43-b223e53fd3b2","Type":"ContainerStarted","Data":"4777da1cb4e77c9b63c9d58fdec5836f32408cf9a9ace70c68da7fcabc2bd62a"}
Jul 23 17:54:27.550040 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:27.549489    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:54:27.550040 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:27.549634    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m5sn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 17:54:27.550040 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:27.549684    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" podUID="d2eb6cca-0902-4227-8a43-b223e53fd3b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 17:54:28.548042 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:28.548004    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m5sn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 17:54:28.549081 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:28.548622    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" podUID="d2eb6cca-0902-4227-8a43-b223e53fd3b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 17:54:28.807624 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:28.807580    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:54:29.548899 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:29.548858    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m5sn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 17:54:29.549486 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:29.549453    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" podUID="d2eb6cca-0902-4227-8a43-b223e53fd3b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 17:54:30.548517 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:30.548476    2582 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-m5sn9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused" start-of-body=
Jul 23 17:54:30.548862 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:30.548832    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9" podUID="d2eb6cca-0902-4227-8a43-b223e53fd3b2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.25:15021/healthz/ready\": dial tcp 10.134.0.25:15021: connect: connection refused"
Jul 23 17:54:31.551631 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:31.551604    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:54:31.552661 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:31.552640    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-m5sn9"
Jul 23 17:54:40.913521 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:40.913474    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:54:57.184860 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:54:57.184824    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:55:02.304003 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:55:02.303228    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:55:09.891205 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:55:09.891165    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:55:41.286454 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:55:41.286370    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:56:06.892675 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.892638    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"]
Jul 23 17:56:06.893173 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.893008    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dd4b0d2-4566-4b30-ab93-9971828fe048" containerName="authorino"
Jul 23 17:56:06.893173 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.893021    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dd4b0d2-4566-4b30-ab93-9971828fe048" containerName="authorino"
Jul 23 17:56:06.893173 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.893041    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="823d198c-023a-47c7-8ebe-32af71d74156" containerName="authorino"
Jul 23 17:56:06.893173 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.893050    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="823d198c-023a-47c7-8ebe-32af71d74156" containerName="authorino"
Jul 23 17:56:06.893173 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.893140    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dd4b0d2-4566-4b30-ab93-9971828fe048" containerName="authorino"
Jul 23 17:56:06.893173 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.893151    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="823d198c-023a-47c7-8ebe-32af71d74156" containerName="authorino"
Jul 23 17:56:06.896190 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.896174    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:06.900365 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.900314    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9tdlq\""
Jul 23 17:56:06.900365 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.900338    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 17:56:06.900608 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.900375    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 23 17:56:06.900608 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.900443    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 17:56:06.905842 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:06.905815    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"]
Jul 23 17:56:07.034328 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.034283    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.034328 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.034324    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.034548 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.034344    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ed1d7d3-98e0-4388-a52d-18a8da279e80-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.034548 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.034435    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jgsd\" (UniqueName: \"kubernetes.io/projected/3ed1d7d3-98e0-4388-a52d-18a8da279e80-kube-api-access-8jgsd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.034548 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.034479    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.034548 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.034516    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.135767 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.135700    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jgsd\" (UniqueName: \"kubernetes.io/projected/3ed1d7d3-98e0-4388-a52d-18a8da279e80-kube-api-access-8jgsd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.135981 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.135780    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.135981 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.135820    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.135981 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.135882    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.135981 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.135908    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.135981 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.135932    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ed1d7d3-98e0-4388-a52d-18a8da279e80-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.136273 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.136250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.136336 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.136305    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.136374 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.136331    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.138175 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.138152    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ed1d7d3-98e0-4388-a52d-18a8da279e80-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.138363 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.138345    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ed1d7d3-98e0-4388-a52d-18a8da279e80-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.146976 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.146909    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jgsd\" (UniqueName: \"kubernetes.io/projected/3ed1d7d3-98e0-4388-a52d-18a8da279e80-kube-api-access-8jgsd\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v\" (UID: \"3ed1d7d3-98e0-4388-a52d-18a8da279e80\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.205777 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.205734    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:07.339949 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.339686    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"]
Jul 23 17:56:07.345271 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.345251    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 17:56:07.667268 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:07.667209    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v" event={"ID":"3ed1d7d3-98e0-4388-a52d-18a8da279e80","Type":"ContainerStarted","Data":"a3ef849f02458a34ad397574b84bb8549d7826b0e0d2c4d384bddff80e06f5fd"}
Jul 23 17:56:08.673292 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:08.673257    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v" event={"ID":"3ed1d7d3-98e0-4388-a52d-18a8da279e80","Type":"ContainerStarted","Data":"4145276beb40d521f30f31c0ab7cde80e8ccb04116cc0f81c118b23922e7a1ec"}
Jul 23 17:56:08.701034 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:08.700936    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v" podStartSLOduration=1.604117688 podStartE2EDuration="2.700917519s" totalImagesPullingTime="1.096799831s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:56:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:56:08.699780266 +0000 UTC m=+663.539288213" watchObservedRunningTime="2026-07-23 17:56:08.700917519 +0000 UTC m=+663.540425446"
Jul 23 17:56:09.390507 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:09.390469    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:56:17.206752 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:17.206699    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:17.207339 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:17.206769    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:17.219358 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:17.219327    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:56:17.733159 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:56:17.733125    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v"
Jul 23 17:57:35.088887 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.088850    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-54bdbf49cd-p4dpg"]
Jul 23 17:57:35.092231 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.092207    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.102891 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.102863    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54bdbf49cd-p4dpg"]
Jul 23 17:57:35.213129 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.213088    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/911b1f50-9941-4abb-8411-85987a0ffb0a-tls-cert\") pod \"authorino-54bdbf49cd-p4dpg\" (UID: \"911b1f50-9941-4abb-8411-85987a0ffb0a\") " pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.213323 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.213233    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8px6t\" (UniqueName: \"kubernetes.io/projected/911b1f50-9941-4abb-8411-85987a0ffb0a-kube-api-access-8px6t\") pod \"authorino-54bdbf49cd-p4dpg\" (UID: \"911b1f50-9941-4abb-8411-85987a0ffb0a\") " pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.314512 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.314469    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8px6t\" (UniqueName: \"kubernetes.io/projected/911b1f50-9941-4abb-8411-85987a0ffb0a-kube-api-access-8px6t\") pod \"authorino-54bdbf49cd-p4dpg\" (UID: \"911b1f50-9941-4abb-8411-85987a0ffb0a\") " pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.314512 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.314512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/911b1f50-9941-4abb-8411-85987a0ffb0a-tls-cert\") pod \"authorino-54bdbf49cd-p4dpg\" (UID: \"911b1f50-9941-4abb-8411-85987a0ffb0a\") " pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.316934 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.316905    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/911b1f50-9941-4abb-8411-85987a0ffb0a-tls-cert\") pod \"authorino-54bdbf49cd-p4dpg\" (UID: \"911b1f50-9941-4abb-8411-85987a0ffb0a\") " pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.323002 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.322977    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8px6t\" (UniqueName: \"kubernetes.io/projected/911b1f50-9941-4abb-8411-85987a0ffb0a-kube-api-access-8px6t\") pod \"authorino-54bdbf49cd-p4dpg\" (UID: \"911b1f50-9941-4abb-8411-85987a0ffb0a\") " pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.400980 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.400895    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-54bdbf49cd-p4dpg"
Jul 23 17:57:35.535278 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:35.535251    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-54bdbf49cd-p4dpg"]
Jul 23 17:57:35.537624 ip-10-0-139-151 kubenswrapper[2582]: W0723 17:57:35.537591    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod911b1f50_9941_4abb_8411_85987a0ffb0a.slice/crio-7668b1b1d8f979f158526716355812ca2f7f88d97b4fa91c0ffcb1c5ca9407a6 WatchSource:0}: Error finding container 7668b1b1d8f979f158526716355812ca2f7f88d97b4fa91c0ffcb1c5ca9407a6: Status 404 returned error can't find the container with id 7668b1b1d8f979f158526716355812ca2f7f88d97b4fa91c0ffcb1c5ca9407a6
Jul 23 17:57:36.059059 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:36.059021    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54bdbf49cd-p4dpg" event={"ID":"911b1f50-9941-4abb-8411-85987a0ffb0a","Type":"ContainerStarted","Data":"7668b1b1d8f979f158526716355812ca2f7f88d97b4fa91c0ffcb1c5ca9407a6"}
Jul 23 17:57:37.065864 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.065822    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-54bdbf49cd-p4dpg" event={"ID":"911b1f50-9941-4abb-8411-85987a0ffb0a","Type":"ContainerStarted","Data":"cd7b4ce6d4045d7a2a3f5d23d345139293c55922ef1943718cc491ba2c848536"}
Jul 23 17:57:37.087763 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.087678    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-54bdbf49cd-p4dpg" podStartSLOduration=1.4726560659999999 podStartE2EDuration="2.087655203s" totalImagesPullingTime="614.999137ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 17:57:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 17:57:37.086286766 +0000 UTC m=+751.925794715" watchObservedRunningTime="2026-07-23 17:57:37.087655203 +0000 UTC m=+751.927163154"
Jul 23 17:57:37.116786 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.116746    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f79ff6667-cqsjf"]
Jul 23 17:57:37.117022 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.116994    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-f79ff6667-cqsjf" podUID="23f67b90-a0ca-4700-b09c-e9155edadfc3" containerName="authorino" containerID="cri-o://f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29" gracePeriod=30
Jul 23 17:57:37.377229 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.377196    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:57:37.533811 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.533779    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23f67b90-a0ca-4700-b09c-e9155edadfc3-tls-cert\") pod \"23f67b90-a0ca-4700-b09c-e9155edadfc3\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") "
Jul 23 17:57:37.534012 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.533847    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9gbw\" (UniqueName: \"kubernetes.io/projected/23f67b90-a0ca-4700-b09c-e9155edadfc3-kube-api-access-q9gbw\") pod \"23f67b90-a0ca-4700-b09c-e9155edadfc3\" (UID: \"23f67b90-a0ca-4700-b09c-e9155edadfc3\") "
Jul 23 17:57:37.535875 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.535843    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23f67b90-a0ca-4700-b09c-e9155edadfc3-kube-api-access-q9gbw" (OuterVolumeSpecName: "kube-api-access-q9gbw") pod "23f67b90-a0ca-4700-b09c-e9155edadfc3" (UID: "23f67b90-a0ca-4700-b09c-e9155edadfc3"). InnerVolumeSpecName "kube-api-access-q9gbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 17:57:37.543166 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.543136    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23f67b90-a0ca-4700-b09c-e9155edadfc3-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "23f67b90-a0ca-4700-b09c-e9155edadfc3" (UID: "23f67b90-a0ca-4700-b09c-e9155edadfc3"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 17:57:37.635111 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.635025    2582 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/23f67b90-a0ca-4700-b09c-e9155edadfc3-tls-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:57:37.635111 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:37.635059    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9gbw\" (UniqueName: \"kubernetes.io/projected/23f67b90-a0ca-4700-b09c-e9155edadfc3-kube-api-access-q9gbw\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 17:57:38.071862 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.071832    2582 generic.go:356] "Generic (PLEG): container finished" podID="23f67b90-a0ca-4700-b09c-e9155edadfc3" containerID="f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29" exitCode=0
Jul 23 17:57:38.072330 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.071868    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f79ff6667-cqsjf"
Jul 23 17:57:38.072330 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.071902    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f79ff6667-cqsjf" event={"ID":"23f67b90-a0ca-4700-b09c-e9155edadfc3","Type":"ContainerDied","Data":"f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29"}
Jul 23 17:57:38.072330 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.071932    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f79ff6667-cqsjf" event={"ID":"23f67b90-a0ca-4700-b09c-e9155edadfc3","Type":"ContainerDied","Data":"3218f3a2502cc76b4f1b3c95f56dc93ed2860760c2b55c5cd976de5336da22c0"}
Jul 23 17:57:38.072330 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.071952    2582 scope.go:117] "RemoveContainer" containerID="f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29"
Jul 23 17:57:38.080962 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.080934    2582 scope.go:117] "RemoveContainer" containerID="f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29"
Jul 23 17:57:38.081300 ip-10-0-139-151 kubenswrapper[2582]: E0723 17:57:38.081268    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29\": container with ID starting with f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29 not found: ID does not exist" containerID="f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29"
Jul 23 17:57:38.081428 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.081303    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29"} err="failed to get container status \"f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29\": rpc error: code = NotFound desc = could not find container \"f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29\": container with ID starting with f5882175cc869a896c18bdb533275781c73b9e44c4fe551b23b358d85b818a29 not found: ID does not exist"
Jul 23 17:57:38.088987 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.088951    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f79ff6667-cqsjf"]
Jul 23 17:57:38.093457 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:38.093422    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f79ff6667-cqsjf"]
Jul 23 17:57:39.783997 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:57:39.783957    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23f67b90-a0ca-4700-b09c-e9155edadfc3" path="/var/lib/kubelet/pods/23f67b90-a0ca-4700-b09c-e9155edadfc3/volumes"
Jul 23 17:58:06.985122 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:58:06.985079    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:58:17.198992 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:58:17.198950    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 17:58:28.393150 ip-10-0-139-151 kubenswrapper[2582]: I0723 17:58:28.393052    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:00:07.298044 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:00:07.298002    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:00:16.192317 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:00:16.192283    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:00:50.597452 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:00:50.597413    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:00:58.494540 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:00:58.494499    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:01:07.687907 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:01:07.687866    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:01:14.991355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:01:14.991300    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:01:25.701729 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:01:25.701672    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:01:35.297454 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:01:35.297360    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:01:46.292016 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:01:46.291978    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:01:56.602585 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:01:56.602547    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:02:21.990213 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:02:21.990174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:02:31.497656 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:02:31.497610    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:02:54.886557 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:02:54.886522    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:03:05.287166 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:03:05.287078    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:03:21.994379 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:03:21.994335    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:03:31.997740 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:03:31.997367    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:03:41.696859 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:03:41.696821    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:03:50.988223 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:03:50.988183    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:04:00.102873 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:04:00.102834    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:05:49.894881 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:05:49.894840    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:05:58.886153 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:05:58.886067    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:06:11.645698 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:06:11.645660    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:06:24.202858 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:06:24.202821    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:06:34.091383 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:06:34.091347    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:06:44.393904 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:06:44.393866    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:06:53.498272 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:06:53.498234    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:07:04.704529 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:07:04.704490    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:07:15.487531 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:07:15.487497    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:07:24.292003 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:07:24.291958    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:07:34.996912 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:07:34.996816    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:07:56.892652 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:07:56.892614    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:08:06.292553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:06.292509    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:08:07.991353 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:07.991309    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"]
Jul 23 18:08:07.991848 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:07.991830    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23f67b90-a0ca-4700-b09c-e9155edadfc3" containerName="authorino"
Jul 23 18:08:07.991901 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:07.991851    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="23f67b90-a0ca-4700-b09c-e9155edadfc3" containerName="authorino"
Jul 23 18:08:07.991968 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:07.991950    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="23f67b90-a0ca-4700-b09c-e9155edadfc3" containerName="authorino"
Jul 23 18:08:07.995179 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:07.995156    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:07.998224 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:07.998192    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-3fd6d150-kserve-self-signed-certs\""
Jul 23 18:08:08.012500 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.012470    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"]
Jul 23 18:08:08.138218 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.138177    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tmp-dir\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.138388 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.138296    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-home\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.138388 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.138375    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-dshm\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.138464 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.138407    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tls-certs\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.138520 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.138505    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-model-cache\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.138556 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.138531    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prqbc\" (UniqueName: \"kubernetes.io/projected/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-kube-api-access-prqbc\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239164 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239129    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tmp-dir\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239188    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-home\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239233    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-dshm\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239255    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tls-certs\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239316    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-model-cache\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239339    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-prqbc\" (UniqueName: \"kubernetes.io/projected/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-kube-api-access-prqbc\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239728 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239666    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tmp-dir\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239728 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239690    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-home\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.239873 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.239771    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-model-cache\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.241768 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.241697    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-dshm\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.241883 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.241866    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tls-certs\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.258534 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.258504    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-prqbc\" (UniqueName: \"kubernetes.io/projected/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-kube-api-access-prqbc\") pod \"e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") " pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.306627 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.306585    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:08.442938 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.442910    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"]
Jul 23 18:08:08.445024 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:08:08.444998    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08b4f008_42c3_44f9_b0b9_646a8f3dcdb7.slice/crio-ef933152051549c011967d834d39848ca0418a3e0c77e2deb15eb2e8efc194e7 WatchSource:0}: Error finding container ef933152051549c011967d834d39848ca0418a3e0c77e2deb15eb2e8efc194e7: Status 404 returned error can't find the container with id ef933152051549c011967d834d39848ca0418a3e0c77e2deb15eb2e8efc194e7
Jul 23 18:08:08.871031 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.870996    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn" event={"ID":"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7","Type":"ContainerStarted","Data":"23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d"}
Jul 23 18:08:08.871031 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.871039    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn" event={"ID":"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7","Type":"ContainerStarted","Data":"ef933152051549c011967d834d39848ca0418a3e0c77e2deb15eb2e8efc194e7"}
Jul 23 18:08:08.892829 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:08.892781    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn" podStartSLOduration=1.892764898 podStartE2EDuration="1.892764898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:08:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:08.891257538 +0000 UTC m=+1383.730765488" watchObservedRunningTime="2026-07-23 18:08:08.892764898 +0000 UTC m=+1383.732272845"
Jul 23 18:08:15.088824 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.088774    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"]
Jul 23 18:08:15.092635 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.092613    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.096503 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.096467    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-3fd6d150-kserve-self-signed-certs\""
Jul 23 18:08:15.107114 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.107081    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"]
Jul 23 18:08:15.207021 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.206965    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tmp-dir\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.207021 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.207031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pjxsp\" (UniqueName: \"kubernetes.io/projected/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-kube-api-access-pjxsp\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.207252 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.207073    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tls-certs\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.207252 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.207156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-dshm\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.207252 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.207190    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-home\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.207252 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.207211    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-model-cache\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308147 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308107    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tmp-dir\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308147 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308150    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pjxsp\" (UniqueName: \"kubernetes.io/projected/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-kube-api-access-pjxsp\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308183    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tls-certs\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308242    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-dshm\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308424 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308367    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-home\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308424 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-model-cache\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308808 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308574    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tmp-dir\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308808 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308746    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-home\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.308968 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.308818    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-model-cache\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.310518 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.310496    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-dshm\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.310818 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.310798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tls-certs\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.326673 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.326650    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pjxsp\" (UniqueName: \"kubernetes.io/projected/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-kube-api-access-pjxsp\") pod \"e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") " pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.402806 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.402719    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:15.540793 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.540698    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"]
Jul 23 18:08:15.542970 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:08:15.542944    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d74b5c3_20ad_47ba_ae52_39d0fd7a604e.slice/crio-4fc6ae749d3caed2f4678d96fa4bdb50fd351bf814f1ee638849ed0588c3e0ae WatchSource:0}: Error finding container 4fc6ae749d3caed2f4678d96fa4bdb50fd351bf814f1ee638849ed0588c3e0ae: Status 404 returned error can't find the container with id 4fc6ae749d3caed2f4678d96fa4bdb50fd351bf814f1ee638849ed0588c3e0ae
Jul 23 18:08:15.904654 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.904611    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6" event={"ID":"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e","Type":"ContainerStarted","Data":"18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393"}
Jul 23 18:08:15.904654 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.904658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6" event={"ID":"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e","Type":"ContainerStarted","Data":"4fc6ae749d3caed2f4678d96fa4bdb50fd351bf814f1ee638849ed0588c3e0ae"}
Jul 23 18:08:15.923168 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:15.923111    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6" podStartSLOduration=0.923092565 podStartE2EDuration="923.092565ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:08:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:08:15.922499179 +0000 UTC m=+1390.762007128" watchObservedRunningTime="2026-07-23 18:08:15.923092565 +0000 UTC m=+1390.762600513"
Jul 23 18:08:18.307261 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:18.307214    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:18.307666 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:18.307304    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:18.319959 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:18.319932    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:18.929233 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:18.929199    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:19.286791 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:19.286690    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:08:25.403413 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:25.403373    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:25.403860 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:25.403436    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:25.415879 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:25.415841    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:25.965941 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:25.965909    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:08:43.626913 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:43.626877    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"]
Jul 23 18:08:44.837003 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:44.836962    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"]
Jul 23 18:08:47.929313 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:47.929274    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:08:58.647342 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:58.647231    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn" podUID="08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" containerName="main" containerID="cri-o://23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d" gracePeriod=45
Jul 23 18:08:58.900185 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:58.900105    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:59.007998 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.007952    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-home\") pod \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") "
Jul 23 18:08:59.007998 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.007994    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-dshm\") pod \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") "
Jul 23 18:08:59.008266 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008045    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tmp-dir\") pod \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") "
Jul 23 18:08:59.008266 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008068    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tls-certs\") pod \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") "
Jul 23 18:08:59.008266 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008128    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-prqbc\" (UniqueName: \"kubernetes.io/projected/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-kube-api-access-prqbc\") pod \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") "
Jul 23 18:08:59.008266 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008163    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-model-cache\") pod \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\" (UID: \"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7\") "
Jul 23 18:08:59.008266 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008243    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-home" (OuterVolumeSpecName: "home") pod "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" (UID: "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:08:59.008524 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008362    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" (UID: "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:08:59.008524 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008478    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-model-cache" (OuterVolumeSpecName: "model-cache") pod "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" (UID: "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:08:59.008524 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008492    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:08:59.008524 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.008509    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:08:59.010456 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.010420    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-kube-api-access-prqbc" (OuterVolumeSpecName: "kube-api-access-prqbc") pod "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" (UID: "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7"). InnerVolumeSpecName "kube-api-access-prqbc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:08:59.010675 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.010653    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-dshm" (OuterVolumeSpecName: "dshm") pod "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" (UID: "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:08:59.010774 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.010655    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" (UID: "08b4f008-42c3-44f9-b0b9-646a8f3dcdb7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:08:59.109002 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.108969    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-prqbc\" (UniqueName: \"kubernetes.io/projected/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-kube-api-access-prqbc\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:08:59.109002 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109000    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:08:59.109002 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109011    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:08:59.109246 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109019    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:08:59.109612 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109583    2582 generic.go:356] "Generic (PLEG): container finished" podID="08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" containerID="23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d" exitCode=0
Jul 23 18:08:59.109819 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109639    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"
Jul 23 18:08:59.109819 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109658    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn" event={"ID":"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7","Type":"ContainerDied","Data":"23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d"}
Jul 23 18:08:59.109819 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109700    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn" event={"ID":"08b4f008-42c3-44f9-b0b9-646a8f3dcdb7","Type":"ContainerDied","Data":"ef933152051549c011967d834d39848ca0418a3e0c77e2deb15eb2e8efc194e7"}
Jul 23 18:08:59.109819 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.109747    2582 scope.go:117] "RemoveContainer" containerID="23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d"
Jul 23 18:08:59.118520 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.118503    2582 scope.go:117] "RemoveContainer" containerID="23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d"
Jul 23 18:08:59.118843 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:08:59.118819    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d\": container with ID starting with 23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d not found: ID does not exist" containerID="23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d"
Jul 23 18:08:59.118912 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.118853    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d"} err="failed to get container status \"23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d\": rpc error: code = NotFound desc = could not find container \"23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d\": container with ID starting with 23374fb3a7ce20bce070687826cfb532fd8bbe7bc26647fa9719b85b9ef0929d not found: ID does not exist"
Jul 23 18:08:59.138286 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.138254    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"]
Jul 23 18:08:59.150859 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.150785    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-3fd6d150-kserve-bbf5b7567-cw2cn"]
Jul 23 18:08:59.785563 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.785520    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" path="/var/lib/kubelet/pods/08b4f008-42c3-44f9-b0b9-646a8f3dcdb7/volumes"
Jul 23 18:08:59.856823 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:08:59.856779    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6" podUID="9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" containerName="main" containerID="cri-o://18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393" gracePeriod=45
Jul 23 18:09:00.105070 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.105046    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:09:00.116056 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.116025    2582 generic.go:356] "Generic (PLEG): container finished" podID="9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" containerID="18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393" exitCode=0
Jul 23 18:09:00.116200 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.116065    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"
Jul 23 18:09:00.116200 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.116065    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6" event={"ID":"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e","Type":"ContainerDied","Data":"18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393"}
Jul 23 18:09:00.116200 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.116108    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6" event={"ID":"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e","Type":"ContainerDied","Data":"4fc6ae749d3caed2f4678d96fa4bdb50fd351bf814f1ee638849ed0588c3e0ae"}
Jul 23 18:09:00.116200 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.116127    2582 scope.go:117] "RemoveContainer" containerID="18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393"
Jul 23 18:09:00.125751 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.125724    2582 scope.go:117] "RemoveContainer" containerID="18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393"
Jul 23 18:09:00.126308 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:09:00.126285    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393\": container with ID starting with 18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393 not found: ID does not exist" containerID="18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393"
Jul 23 18:09:00.126369 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.126321    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393"} err="failed to get container status \"18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393\": rpc error: code = NotFound desc = could not find container \"18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393\": container with ID starting with 18d1bcc2c6184e236c3d44da711565e386b010cfb3649c4a15ff4e0e14373393 not found: ID does not exist"
Jul 23 18:09:00.220315 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220274    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pjxsp\" (UniqueName: \"kubernetes.io/projected/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-kube-api-access-pjxsp\") pod \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") "
Jul 23 18:09:00.220315 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220323    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-model-cache\") pod \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") "
Jul 23 18:09:00.220540 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220353    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-dshm\") pod \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") "
Jul 23 18:09:00.220540 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220431    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tmp-dir\") pod \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") "
Jul 23 18:09:00.220540 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220448    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-home\") pod \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") "
Jul 23 18:09:00.220540 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220486    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tls-certs\") pod \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\" (UID: \"9d74b5c3-20ad-47ba-ae52-39d0fd7a604e\") "
Jul 23 18:09:00.220786 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220626    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-model-cache" (OuterVolumeSpecName: "model-cache") pod "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" (UID: "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:09:00.220786 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220736    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" (UID: "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:09:00.220786 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220778    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:09:00.220923 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.220794    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-home" (OuterVolumeSpecName: "home") pod "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" (UID: "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:09:00.222480 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.222456    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" (UID: "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:09:00.222562 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.222530    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-kube-api-access-pjxsp" (OuterVolumeSpecName: "kube-api-access-pjxsp") pod "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" (UID: "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e"). InnerVolumeSpecName "kube-api-access-pjxsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:09:00.222562 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.222553    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-dshm" (OuterVolumeSpecName: "dshm") pod "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" (UID: "9d74b5c3-20ad-47ba-ae52-39d0fd7a604e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:09:00.322059 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.322013    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:09:00.322059 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.322049    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:09:00.322059 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.322062    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:09:00.322301 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.322072    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:09:00.322301 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.322084    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pjxsp\" (UniqueName: \"kubernetes.io/projected/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e-kube-api-access-pjxsp\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:09:00.452224 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.452185    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"]
Jul 23 18:09:00.457915 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:00.457879    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-3fd6d150-kserve-65dbcbd479-l2hf6"]
Jul 23 18:09:01.783823 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:01.783791    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" path="/var/lib/kubelet/pods/9d74b5c3-20ad-47ba-ae52-39d0fd7a604e/volumes"
Jul 23 18:09:03.249992 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:03.249950    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:09:12.584160 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:12.584120    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:09:24.485778 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:24.485734    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:09:35.084470 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:35.084431    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:09:44.781318 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:44.781285    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:09:54.992634 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:09:54.992596    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:10:04.890338 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:10:04.890296    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:10:14.292080 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:10:14.292040    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:10:27.604150 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:10:27.604062    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:10:36.395858 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:10:36.395820    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:10:46.795199 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:10:46.795162    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:10:55.813465 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:10:55.813428    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:11:08.289541 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:11:08.289498    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:11:17.657459 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:11:17.657422    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:11:29.887295 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:11:29.887254    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:11:40.788691 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:11:40.788651    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:11:49.701357 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:11:49.701323    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:11:55.784973 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:11:55.784933    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:12:09.189392 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:09.189304    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:12:23.885279 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:23.885245    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:12:29.745039 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.745008    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"]
Jul 23 18:12:29.745645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.745315    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" containerName="main"
Jul 23 18:12:29.745645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.745327    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" containerName="main"
Jul 23 18:12:29.745645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.745342    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" containerName="main"
Jul 23 18:12:29.745645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.745348    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" containerName="main"
Jul 23 18:12:29.745645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.745398    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="08b4f008-42c3-44f9-b0b9-646a8f3dcdb7" containerName="main"
Jul 23 18:12:29.745645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.745407    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d74b5c3-20ad-47ba-ae52-39d0fd7a604e" containerName="main"
Jul 23 18:12:29.748409 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.748391    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.752332 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.752309    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-60a07bef-openshift-default-dockercfg-6zfwj\""
Jul 23 18:12:29.786586 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.786548    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"]
Jul 23 18:12:29.929458 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929422    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dbac9d-34e9-482c-adf2-ace66bf02050-istiod-ca-cert\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.929458 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929461    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdltc\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-kube-api-access-fdltc\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.929685 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929582    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-podinfo\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.929685 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929607    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-socket\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.929685 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929668    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-envoy\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.929831 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929700    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-token\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.929831 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929778    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-credential-socket\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.929831 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929819    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-data\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:29.930071 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:29.929839    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-certs\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030416 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030307    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-credential-socket\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030416 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030356    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-data\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030416 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030378    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-certs\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030416 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030396    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dbac9d-34e9-482c-adf2-ace66bf02050-istiod-ca-cert\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030796 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdltc\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-kube-api-access-fdltc\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030796 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030497    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-podinfo\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030796 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030515    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-socket\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030796 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030545    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-envoy\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.030796 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030569    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-token\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.031195 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030807    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-data\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.031195 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.030989    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-credential-socket\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.031195 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.031021    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-certs\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.031195 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.031108    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-socket\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.031423 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.031400    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dbac9d-34e9-482c-adf2-ace66bf02050-istiod-ca-cert\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.033121 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.033089    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-envoy\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.033121 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.033114    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-podinfo\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.040248 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.040222    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-token\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.040572 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.040550    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdltc\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-kube-api-access-fdltc\") pod \"e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") " pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.058103 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.058078    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:30.198081 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.198055    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"]
Jul 23 18:12:30.200747 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:12:30.200695    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29dbac9d_34e9_482c_adf2_ace66bf02050.slice/crio-6395b4ef7d16ab01f0e1428a16192d55e941112dfac14cd516e3455b2a5a5371 WatchSource:0}: Error finding container 6395b4ef7d16ab01f0e1428a16192d55e941112dfac14cd516e3455b2a5a5371: Status 404 returned error can't find the container with id 6395b4ef7d16ab01f0e1428a16192d55e941112dfac14cd516e3455b2a5a5371
Jul 23 18:12:30.203148 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.203114    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 18:12:30.203241 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.203182    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 18:12:30.203241 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:30.203210    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 18:12:31.065796 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:31.065758    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" event={"ID":"29dbac9d-34e9-482c-adf2-ace66bf02050","Type":"ContainerStarted","Data":"b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794"}
Jul 23 18:12:31.066203 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:31.065818    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" event={"ID":"29dbac9d-34e9-482c-adf2-ace66bf02050","Type":"ContainerStarted","Data":"6395b4ef7d16ab01f0e1428a16192d55e941112dfac14cd516e3455b2a5a5371"}
Jul 23 18:12:31.091583 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:31.091517    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" podStartSLOduration=2.091495494 podStartE2EDuration="2.091495494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:12:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:12:31.091108906 +0000 UTC m=+1645.930616855" watchObservedRunningTime="2026-07-23 18:12:31.091495494 +0000 UTC m=+1645.931003442"
Jul 23 18:12:32.059113 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:32.059071    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:32.063730 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:32.063694    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:32.070567 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:32.070542    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:32.071446 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:32.071427    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:37.690867 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:37.690823    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:12:46.135628 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:46.135593    2582 generic.go:356] "Generic (PLEG): container finished" podID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerID="b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794" exitCode=0
Jul 23 18:12:46.136086 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:46.135668    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" event={"ID":"29dbac9d-34e9-482c-adf2-ace66bf02050","Type":"ContainerDied","Data":"b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794"}
Jul 23 18:12:46.136086 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:46.136062    2582 scope.go:117] "RemoveContainer" containerID="b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794"
Jul 23 18:12:46.137011 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:46.136987    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 18:12:46.137068 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:46.137050    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 18:12:46.137105 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:46.137090    2582 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 18:12:47.070637 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:47.070588    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:47.145102 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:47.145048    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" event={"ID":"29dbac9d-34e9-482c-adf2-ace66bf02050","Type":"ContainerStarted","Data":"54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923"}
Jul 23 18:12:48.059064 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:48.059023    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:48.060266 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:48.060235    2582 patch_prober.go:28] interesting pod/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 18:12:48.060438 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:48.060302    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 18:12:49.059215 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:49.059166    2582 patch_prober.go:28] interesting pod/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 18:12:49.059632 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:49.059245    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 18:12:50.059533 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:50.059489    2582 patch_prober.go:28] interesting pod/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 18:12:50.059966 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:50.059589    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 18:12:51.059036 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:51.058984    2582 patch_prober.go:28] interesting pod/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused" start-of-body=
Jul 23 18:12:51.059230 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:51.059076    2582 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.35:15021/healthz/ready\": dial tcp 10.134.0.35:15021: connect: connection refused"
Jul 23 18:12:52.062962 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:52.062931    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:52.063375 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:52.063213    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:12:52.063891 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:12:52.063872    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:13:11.489888 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:11.489837    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:13:13.297761 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:13.297723    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"]
Jul 23 18:13:13.298179 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:13.297959    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy" containerID="cri-o://54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923" gracePeriod=30
Jul 23 18:13:18.548392 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.548367    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:13:18.597880 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.597775    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdltc\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-kube-api-access-fdltc\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.597880 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.597833    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-credential-socket\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.597880 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.597868    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-certs\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.598191 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.597934    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-podinfo\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.598191 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.597961    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-envoy\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.598191 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.597995    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-data\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.598191 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598021    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-token\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.598191 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598050    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-socket\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.598191 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598119    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dbac9d-34e9-482c-adf2-ace66bf02050-istiod-ca-cert\") pod \"29dbac9d-34e9-482c-adf2-ace66bf02050\" (UID: \"29dbac9d-34e9-482c-adf2-ace66bf02050\") "
Jul 23 18:13:18.598523 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598184    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:13:18.598523 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598210    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:13:18.598523 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598389    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:13:18.598523 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598454    2582 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-credential-socket\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.598523 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598474    2582 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.598813 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.598625    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29dbac9d-34e9-482c-adf2-ace66bf02050-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 18:13:18.599095 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.599066    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-data" (OuterVolumeSpecName: "istio-data") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:13:18.600409 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.600385    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 18:13:18.600526 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.600488    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:13:18.600526 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.600491    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-kube-api-access-fdltc" (OuterVolumeSpecName: "kube-api-access-fdltc") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "kube-api-access-fdltc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:13:18.600642 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.600531    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-token" (OuterVolumeSpecName: "istio-token") pod "29dbac9d-34e9-482c-adf2-ace66bf02050" (UID: "29dbac9d-34e9-482c-adf2-ace66bf02050"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:13:18.699642 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.699604    2582 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-podinfo\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.699642 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.699633    2582 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-envoy\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.699642 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.699641    2582 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-data\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.699642 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.699653    2582 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-istio-token\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.699949 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.699660    2582 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29dbac9d-34e9-482c-adf2-ace66bf02050-workload-socket\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.699949 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.699669    2582 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29dbac9d-34e9-482c-adf2-ace66bf02050-istiod-ca-cert\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:18.699949 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:18.699679    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdltc\" (UniqueName: \"kubernetes.io/projected/29dbac9d-34e9-482c-adf2-ace66bf02050-kube-api-access-fdltc\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:13:19.300492 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.300456    2582 generic.go:356] "Generic (PLEG): container finished" podID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerID="54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923" exitCode=0
Jul 23 18:13:19.300727 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.300517    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"
Jul 23 18:13:19.300727 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.300542    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" event={"ID":"29dbac9d-34e9-482c-adf2-ace66bf02050","Type":"ContainerDied","Data":"54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923"}
Jul 23 18:13:19.300727 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.300594    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m" event={"ID":"29dbac9d-34e9-482c-adf2-ace66bf02050","Type":"ContainerDied","Data":"6395b4ef7d16ab01f0e1428a16192d55e941112dfac14cd516e3455b2a5a5371"}
Jul 23 18:13:19.300727 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.300618    2582 scope.go:117] "RemoveContainer" containerID="54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923"
Jul 23 18:13:19.309897 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.309876    2582 scope.go:117] "RemoveContainer" containerID="b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794"
Jul 23 18:13:19.318060 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.318042    2582 scope.go:117] "RemoveContainer" containerID="54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923"
Jul 23 18:13:19.318370 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:13:19.318352    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923\": container with ID starting with 54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923 not found: ID does not exist" containerID="54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923"
Jul 23 18:13:19.318410 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.318381    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923"} err="failed to get container status \"54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923\": rpc error: code = NotFound desc = could not find container \"54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923\": container with ID starting with 54a421286694e9abdace078f86e3e904f0f44d59f873d7abdfa319b010701923 not found: ID does not exist"
Jul 23 18:13:19.318410 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.318403    2582 scope.go:117] "RemoveContainer" containerID="b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794"
Jul 23 18:13:19.318659 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:13:19.318641    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794\": container with ID starting with b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794 not found: ID does not exist" containerID="b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794"
Jul 23 18:13:19.318699 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.318666    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794"} err="failed to get container status \"b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794\": rpc error: code = NotFound desc = could not find container \"b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794\": container with ID starting with b79acf60dba408f98c14ef78ffec5207d72d6994faa6bfc08f128148802a8794 not found: ID does not exist"
Jul 23 18:13:19.326103 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.326074    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"]
Jul 23 18:13:19.329576 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.329548    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-60a07bef-openshift-default-7c6ccb688f-jpm9m"]
Jul 23 18:13:19.783663 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:19.783631    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" path="/var/lib/kubelet/pods/29dbac9d-34e9-482c-adf2-ace66bf02050/volumes"
Jul 23 18:13:23.486559 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:23.486525    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:13:58.701509 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:13:58.701469    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:14:09.689609 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:14:09.689570    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:14:41.289844 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:14:41.289797    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:14:53.186728 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:14:53.186679    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:15:26.891237 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:15:26.891201    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:16:52.786832 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:16:52.786749    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:17:01.961531 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:17:01.961493    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:17:10.687109 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:17:10.687073    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:17:21.282000 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:17:21.281962    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:17:37.084998 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:17:37.084961    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:18:12.201331 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:12.201248    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:18:13.761612 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.761573    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"]
Jul 23 18:18:13.762027 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.761905    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy"
Jul 23 18:18:13.762027 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.761916    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy"
Jul 23 18:18:13.762027 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.761939    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy"
Jul 23 18:18:13.762027 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.761945    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy"
Jul 23 18:18:13.762027 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.761997    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy"
Jul 23 18:18:13.762027 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.762003    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="29dbac9d-34e9-482c-adf2-ace66bf02050" containerName="istio-proxy"
Jul 23 18:18:13.765553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.765536    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:13.769245 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.769222    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-def6e9dd\"/\"kube-root-ca.crt\""
Jul 23 18:18:13.769245 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.769234    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-def6e9dd\"/\"default-dockercfg-wl5sf\""
Jul 23 18:18:13.769460 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.769222    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-def6e9dd\"/\"openshift-service-ca.crt\""
Jul 23 18:18:13.769460 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.769253    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-def6e9dd\"/\"e2e-lifecycle-model-def6e9dd-kserve-self-signed-certs\""
Jul 23 18:18:13.775055 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.775030    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"]
Jul 23 18:18:13.910197 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.910159    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-tmp-dir\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:13.910413 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.910284    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkb64\" (UniqueName: \"kubernetes.io/projected/c310e256-3b46-4130-b666-cb863ca239ce-kube-api-access-zkb64\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:13.910413 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.910337    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-model-cache\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:13.910413 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.910370    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-home\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:13.910413 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.910398    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-dshm\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:13.910582 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:13.910523    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c310e256-3b46-4130-b666-cb863ca239ce-tls-certs\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011289 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011244    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkb64\" (UniqueName: \"kubernetes.io/projected/c310e256-3b46-4130-b666-cb863ca239ce-kube-api-access-zkb64\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011472 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-model-cache\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011472 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011358    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-home\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011472 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-dshm\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011472 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011409    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c310e256-3b46-4130-b666-cb863ca239ce-tls-certs\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011782 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011477    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-tmp-dir\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011860 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011786    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-model-cache\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011919 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011862    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-home\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.011988 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.011953    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-tmp-dir\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.013630 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.013609    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-dshm\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.013920 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.013901    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c310e256-3b46-4130-b666-cb863ca239ce-tls-certs\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.022073 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.022044    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkb64\" (UniqueName: \"kubernetes.io/projected/c310e256-3b46-4130-b666-cb863ca239ce-kube-api-access-zkb64\") pod \"e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") " pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.074827 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.074790    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:14.205961 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.205934    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"]
Jul 23 18:18:14.209529 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:18:14.209498    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc310e256_3b46_4130_b666_cb863ca239ce.slice/crio-8ecd5a26a43af36ec863cb7463f5b30fdfd69a0ed0e057cfdbcb3be3d3cc38a5 WatchSource:0}: Error finding container 8ecd5a26a43af36ec863cb7463f5b30fdfd69a0ed0e057cfdbcb3be3d3cc38a5: Status 404 returned error can't find the container with id 8ecd5a26a43af36ec863cb7463f5b30fdfd69a0ed0e057cfdbcb3be3d3cc38a5
Jul 23 18:18:14.615632 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.615597    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9" event={"ID":"c310e256-3b46-4130-b666-cb863ca239ce","Type":"ContainerStarted","Data":"fec767c0c94086f9a3590e70f4c73361285e5ac3ab37541f2322f89f46de4c39"}
Jul 23 18:18:14.615632 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.615638    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9" event={"ID":"c310e256-3b46-4130-b666-cb863ca239ce","Type":"ContainerStarted","Data":"8ecd5a26a43af36ec863cb7463f5b30fdfd69a0ed0e057cfdbcb3be3d3cc38a5"}
Jul 23 18:18:14.636017 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:14.635968    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9" podStartSLOduration=1.635951382 podStartE2EDuration="1.635951382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:18:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:18:14.634325063 +0000 UTC m=+1989.473833022" watchObservedRunningTime="2026-07-23 18:18:14.635951382 +0000 UTC m=+1989.475459386"
Jul 23 18:18:24.075455 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:24.075417    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:24.075955 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:24.075467    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:24.088583 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:24.088557    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:24.283138 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:24.283098    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:18:24.671393 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:24.671362    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:18:47.193847 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:18:47.193803    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:19:04.622152 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:04.622112    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:19:13.875901 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:13.875859    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:19:17.834011 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:17.833970    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"]
Jul 23 18:19:32.856117 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:32.856026    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9" podUID="c310e256-3b46-4130-b666-cb863ca239ce" containerName="main" containerID="cri-o://fec767c0c94086f9a3590e70f4c73361285e5ac3ab37541f2322f89f46de4c39" gracePeriod=45
Jul 23 18:19:32.978005 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:32.977968    2582 generic.go:356] "Generic (PLEG): container finished" podID="c310e256-3b46-4130-b666-cb863ca239ce" containerID="fec767c0c94086f9a3590e70f4c73361285e5ac3ab37541f2322f89f46de4c39" exitCode=0
Jul 23 18:19:32.978151 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:32.978036    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9" event={"ID":"c310e256-3b46-4130-b666-cb863ca239ce","Type":"ContainerDied","Data":"fec767c0c94086f9a3590e70f4c73361285e5ac3ab37541f2322f89f46de4c39"}
Jul 23 18:19:33.108408 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.108337    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:19:33.217206 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217162    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-tmp-dir\") pod \"c310e256-3b46-4130-b666-cb863ca239ce\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") "
Jul 23 18:19:33.217382 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217252    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zkb64\" (UniqueName: \"kubernetes.io/projected/c310e256-3b46-4130-b666-cb863ca239ce-kube-api-access-zkb64\") pod \"c310e256-3b46-4130-b666-cb863ca239ce\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") "
Jul 23 18:19:33.217382 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217271    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-model-cache\") pod \"c310e256-3b46-4130-b666-cb863ca239ce\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") "
Jul 23 18:19:33.217382 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217287    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-home\") pod \"c310e256-3b46-4130-b666-cb863ca239ce\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") "
Jul 23 18:19:33.217382 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217305    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c310e256-3b46-4130-b666-cb863ca239ce-tls-certs\") pod \"c310e256-3b46-4130-b666-cb863ca239ce\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") "
Jul 23 18:19:33.217382 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217326    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-dshm\") pod \"c310e256-3b46-4130-b666-cb863ca239ce\" (UID: \"c310e256-3b46-4130-b666-cb863ca239ce\") "
Jul 23 18:19:33.217631 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217446    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c310e256-3b46-4130-b666-cb863ca239ce" (UID: "c310e256-3b46-4130-b666-cb863ca239ce"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:19:33.217631 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217542    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-model-cache" (OuterVolumeSpecName: "model-cache") pod "c310e256-3b46-4130-b666-cb863ca239ce" (UID: "c310e256-3b46-4130-b666-cb863ca239ce"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:19:33.217631 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217555    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-home" (OuterVolumeSpecName: "home") pod "c310e256-3b46-4130-b666-cb863ca239ce" (UID: "c310e256-3b46-4130-b666-cb863ca239ce"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:19:33.217631 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.217626    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:19:33.219635 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.219605    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c310e256-3b46-4130-b666-cb863ca239ce-kube-api-access-zkb64" (OuterVolumeSpecName: "kube-api-access-zkb64") pod "c310e256-3b46-4130-b666-cb863ca239ce" (UID: "c310e256-3b46-4130-b666-cb863ca239ce"). InnerVolumeSpecName "kube-api-access-zkb64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:19:33.219774 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.219645    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c310e256-3b46-4130-b666-cb863ca239ce-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c310e256-3b46-4130-b666-cb863ca239ce" (UID: "c310e256-3b46-4130-b666-cb863ca239ce"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:19:33.219774 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.219688    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-dshm" (OuterVolumeSpecName: "dshm") pod "c310e256-3b46-4130-b666-cb863ca239ce" (UID: "c310e256-3b46-4130-b666-cb863ca239ce"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:19:33.318880 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.318842    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zkb64\" (UniqueName: \"kubernetes.io/projected/c310e256-3b46-4130-b666-cb863ca239ce-kube-api-access-zkb64\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:19:33.318880 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.318873    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:19:33.318880 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.318882    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:19:33.318880 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.318890    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c310e256-3b46-4130-b666-cb863ca239ce-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:19:33.319152 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.318898    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c310e256-3b46-4130-b666-cb863ca239ce-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:19:33.984254 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.984225    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"
Jul 23 18:19:33.984754 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.984268    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9" event={"ID":"c310e256-3b46-4130-b666-cb863ca239ce","Type":"ContainerDied","Data":"8ecd5a26a43af36ec863cb7463f5b30fdfd69a0ed0e057cfdbcb3be3d3cc38a5"}
Jul 23 18:19:33.984754 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:33.984322    2582 scope.go:117] "RemoveContainer" containerID="fec767c0c94086f9a3590e70f4c73361285e5ac3ab37541f2322f89f46de4c39"
Jul 23 18:19:34.000386 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:34.000353    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"]
Jul 23 18:19:34.001818 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:34.001787    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-def6e9dd/e2e-lifecycle-model-def6e9dd-kserve-85674f55ff-82jj9"]
Jul 23 18:19:35.783758 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:19:35.783702    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c310e256-3b46-4130-b666-cb863ca239ce" path="/var/lib/kubelet/pods/c310e256-3b46-4130-b666-cb863ca239ce/volumes"
Jul 23 18:20:04.882062 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:20:04.882027    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:20:14.501686 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:20:14.501644    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:21:06.380631 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:06.380534    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:21:19.185866 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:19.185829    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:21:23.282493 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.282454    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"]
Jul 23 18:21:23.282918 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.282831    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c310e256-3b46-4130-b666-cb863ca239ce" containerName="main"
Jul 23 18:21:23.282918 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.282844    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="c310e256-3b46-4130-b666-cb863ca239ce" containerName="main"
Jul 23 18:21:23.282918 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.282902    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="c310e256-3b46-4130-b666-cb863ca239ce" containerName="main"
Jul 23 18:21:23.286358 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.286341    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.289890 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.289864    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-090448\"/\"kube-root-ca.crt\""
Jul 23 18:21:23.290034 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.289889    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-090448\"/\"test-model-090448-kserve-self-signed-certs\""
Jul 23 18:21:23.290034 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.289901    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-090448\"/\"default-dockercfg-t7nd8\""
Jul 23 18:21:23.290034 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.289907    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-090448\"/\"openshift-service-ca.crt\""
Jul 23 18:21:23.295777 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.295754    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"]
Jul 23 18:21:23.415645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.415604    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-dshm\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.415645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.415649    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7nrk\" (UniqueName: \"kubernetes.io/projected/354815f9-26d1-43d3-98ce-2fb7cdc62c83-kube-api-access-l7nrk\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.415891 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.415672    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tls-certs\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.415891 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.415790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-home\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.415891 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.415830    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tmp-dir\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.415891 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.415864    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-model-cache\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.516589 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.516546    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-dshm\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.516589 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.516587    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7nrk\" (UniqueName: \"kubernetes.io/projected/354815f9-26d1-43d3-98ce-2fb7cdc62c83-kube-api-access-l7nrk\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.516869 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.516619    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tls-certs\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.516869 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.516661    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-home\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.516869 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.516696    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tmp-dir\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.516869 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.516765    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-model-cache\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.517138 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.517100    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-home\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.517138 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.517119    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-model-cache\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.517280 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.517190    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tmp-dir\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.518974 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.518951    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-dshm\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.519167 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.519152    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tls-certs\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.524467 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.524449    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7nrk\" (UniqueName: \"kubernetes.io/projected/354815f9-26d1-43d3-98ce-2fb7cdc62c83-kube-api-access-l7nrk\") pod \"test-model-090448-kserve-655b6ccdf5-bhlwz\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") " pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.596676 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.596629    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:23.735473 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:23.735423    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"]
Jul 23 18:21:23.737642 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:21:23.737615    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod354815f9_26d1_43d3_98ce_2fb7cdc62c83.slice/crio-800e3754c05a8c98e820a8941972ec61258caf781254539f1d5a9d348a00828b WatchSource:0}: Error finding container 800e3754c05a8c98e820a8941972ec61258caf781254539f1d5a9d348a00828b: Status 404 returned error can't find the container with id 800e3754c05a8c98e820a8941972ec61258caf781254539f1d5a9d348a00828b
Jul 23 18:21:24.476653 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:24.476613    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz" event={"ID":"354815f9-26d1-43d3-98ce-2fb7cdc62c83","Type":"ContainerStarted","Data":"64702ad444e68494ce7a7e64814fc5e8a135e441e3ff3a3a9b43b41a25173687"}
Jul 23 18:21:24.476653 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:24.476657    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz" event={"ID":"354815f9-26d1-43d3-98ce-2fb7cdc62c83","Type":"ContainerStarted","Data":"800e3754c05a8c98e820a8941972ec61258caf781254539f1d5a9d348a00828b"}
Jul 23 18:21:24.498069 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:24.498012    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz" podStartSLOduration=1.497997211 podStartE2EDuration="1.497997211s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:21:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:21:24.496308925 +0000 UTC m=+2179.335816886" watchObservedRunningTime="2026-07-23 18:21:24.497997211 +0000 UTC m=+2179.337505186"
Jul 23 18:21:27.183117 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:27.183074    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:21:33.596976 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:33.596936    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:33.597409 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:33.597031    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:33.610577 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:33.610540    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:34.533267 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:34.533235    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:21:43.093562 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.093525    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"]
Jul 23 18:21:43.096796 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.096774    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.100494 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.100473    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7893d7\"/\"kube-root-ca.crt\""
Jul 23 18:21:43.100627 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.100495    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7893d7\"/\"test-model-7893d7-kserve-self-signed-certs\""
Jul 23 18:21:43.100774 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.100758    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7893d7\"/\"default-dockercfg-8dbj2\""
Jul 23 18:21:43.100867 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.100761    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7893d7\"/\"openshift-service-ca.crt\""
Jul 23 18:21:43.108974 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.108952    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"]
Jul 23 18:21:43.184795 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.184751    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-dshm\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.184795 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.184789    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-model-cache\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.185055 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.184823    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-home\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.185055 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.184850    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tmp-dir\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.185055 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.184865    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tls-certs\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.185055 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.184986    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z7x8\" (UniqueName: \"kubernetes.io/projected/cb9b386e-e4ed-4633-a69a-15ed116a48f6-kube-api-access-6z7x8\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286077 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.285915    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-dshm\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286278 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286093    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-model-cache\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286278 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-home\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286278 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tmp-dir\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286278 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286202    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tls-certs\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286278 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286269    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z7x8\" (UniqueName: \"kubernetes.io/projected/cb9b386e-e4ed-4633-a69a-15ed116a48f6-kube-api-access-6z7x8\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286539 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286497    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-model-cache\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286789 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286776    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tmp-dir\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.286882 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.286863    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-home\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.288861 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.288833    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-dshm\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.289405 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.289386    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tls-certs\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.296192 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.296167    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z7x8\" (UniqueName: \"kubernetes.io/projected/cb9b386e-e4ed-4633-a69a-15ed116a48f6-kube-api-access-6z7x8\") pod \"test-model-7893d7-kserve-5955f985-97jdm\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") " pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.406463 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.406371    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:43.543975 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.543942    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"]
Jul 23 18:21:43.546652 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:21:43.546623    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb9b386e_e4ed_4633_a69a_15ed116a48f6.slice/crio-4e5b5a10fe2bd8329b5c43eaebbbe61d70c6d1e9a4cd96176382467db964f5cc WatchSource:0}: Error finding container 4e5b5a10fe2bd8329b5c43eaebbbe61d70c6d1e9a4cd96176382467db964f5cc: Status 404 returned error can't find the container with id 4e5b5a10fe2bd8329b5c43eaebbbe61d70c6d1e9a4cd96176382467db964f5cc
Jul 23 18:21:43.565997 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:43.565969    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm" event={"ID":"cb9b386e-e4ed-4633-a69a-15ed116a48f6","Type":"ContainerStarted","Data":"4e5b5a10fe2bd8329b5c43eaebbbe61d70c6d1e9a4cd96176382467db964f5cc"}
Jul 23 18:21:44.572040 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:44.571999    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm" event={"ID":"cb9b386e-e4ed-4633-a69a-15ed116a48f6","Type":"ContainerStarted","Data":"f7ad6396f1bf6a406021b0906ed53b3d2cdce40a8d0517ecb92f9b8b3c0afbfc"}
Jul 23 18:21:44.590064 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:44.590013    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm" podStartSLOduration=1.5899947330000002 podStartE2EDuration="1.589994733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:21:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:21:44.589089217 +0000 UTC m=+2199.428597169" watchObservedRunningTime="2026-07-23 18:21:44.589994733 +0000 UTC m=+2199.429502681"
Jul 23 18:21:47.582215 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:47.582174    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:21:53.407393 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:53.407355    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:53.407886 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:53.407412    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:53.419959 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:53.419929    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:21:53.625025 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:21:53.624993    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:22:05.974036 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:05.974001    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:22:37.643818 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:37.643736    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"]
Jul 23 18:22:38.667754 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:38.667694    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"]
Jul 23 18:22:42.980852 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:42.980811    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:22:52.663544 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:52.663465    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz" podUID="354815f9-26d1-43d3-98ce-2fb7cdc62c83" containerName="main" containerID="cri-o://64702ad444e68494ce7a7e64814fc5e8a135e441e3ff3a3a9b43b41a25173687" gracePeriod=45
Jul 23 18:22:52.879106 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:52.879066    2582 generic.go:356] "Generic (PLEG): container finished" podID="354815f9-26d1-43d3-98ce-2fb7cdc62c83" containerID="64702ad444e68494ce7a7e64814fc5e8a135e441e3ff3a3a9b43b41a25173687" exitCode=0
Jul 23 18:22:52.879274 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:52.879141    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz" event={"ID":"354815f9-26d1-43d3-98ce-2fb7cdc62c83","Type":"ContainerDied","Data":"64702ad444e68494ce7a7e64814fc5e8a135e441e3ff3a3a9b43b41a25173687"}
Jul 23 18:22:52.914068 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:52.914004    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:22:53.032417 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032377    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-dshm\") pod \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") "
Jul 23 18:22:53.032619 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032431    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-home\") pod \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") "
Jul 23 18:22:53.032619 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032473    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tmp-dir\") pod \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") "
Jul 23 18:22:53.032619 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032506    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-model-cache\") pod \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") "
Jul 23 18:22:53.032619 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032543    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tls-certs\") pod \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") "
Jul 23 18:22:53.032619 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032598    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7nrk\" (UniqueName: \"kubernetes.io/projected/354815f9-26d1-43d3-98ce-2fb7cdc62c83-kube-api-access-l7nrk\") pod \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\" (UID: \"354815f9-26d1-43d3-98ce-2fb7cdc62c83\") "
Jul 23 18:22:53.032907 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032815    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "354815f9-26d1-43d3-98ce-2fb7cdc62c83" (UID: "354815f9-26d1-43d3-98ce-2fb7cdc62c83"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:53.032907 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032832    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-home" (OuterVolumeSpecName: "home") pod "354815f9-26d1-43d3-98ce-2fb7cdc62c83" (UID: "354815f9-26d1-43d3-98ce-2fb7cdc62c83"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:53.032907 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.032807    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-model-cache" (OuterVolumeSpecName: "model-cache") pod "354815f9-26d1-43d3-98ce-2fb7cdc62c83" (UID: "354815f9-26d1-43d3-98ce-2fb7cdc62c83"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:53.034780 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.034748    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-dshm" (OuterVolumeSpecName: "dshm") pod "354815f9-26d1-43d3-98ce-2fb7cdc62c83" (UID: "354815f9-26d1-43d3-98ce-2fb7cdc62c83"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:53.034890 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.034771    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "354815f9-26d1-43d3-98ce-2fb7cdc62c83" (UID: "354815f9-26d1-43d3-98ce-2fb7cdc62c83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:22:53.034890 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.034800    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/354815f9-26d1-43d3-98ce-2fb7cdc62c83-kube-api-access-l7nrk" (OuterVolumeSpecName: "kube-api-access-l7nrk") pod "354815f9-26d1-43d3-98ce-2fb7cdc62c83" (UID: "354815f9-26d1-43d3-98ce-2fb7cdc62c83"). InnerVolumeSpecName "kube-api-access-l7nrk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:22:53.133772 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.133738    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:53.133772 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.133767    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7nrk\" (UniqueName: \"kubernetes.io/projected/354815f9-26d1-43d3-98ce-2fb7cdc62c83-kube-api-access-l7nrk\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:53.133772 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.133777    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:53.134024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.133787    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:53.134024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.133796    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:53.134024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.133805    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/354815f9-26d1-43d3-98ce-2fb7cdc62c83-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:53.688076 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.688016    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm" podUID="cb9b386e-e4ed-4633-a69a-15ed116a48f6" containerName="main" containerID="cri-o://f7ad6396f1bf6a406021b0906ed53b3d2cdce40a8d0517ecb92f9b8b3c0afbfc" gracePeriod=45
Jul 23 18:22:53.886963 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.886926    2582 generic.go:356] "Generic (PLEG): container finished" podID="cb9b386e-e4ed-4633-a69a-15ed116a48f6" containerID="f7ad6396f1bf6a406021b0906ed53b3d2cdce40a8d0517ecb92f9b8b3c0afbfc" exitCode=0
Jul 23 18:22:53.887133 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.886982    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm" event={"ID":"cb9b386e-e4ed-4633-a69a-15ed116a48f6","Type":"ContainerDied","Data":"f7ad6396f1bf6a406021b0906ed53b3d2cdce40a8d0517ecb92f9b8b3c0afbfc"}
Jul 23 18:22:53.888553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.888528    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz" event={"ID":"354815f9-26d1-43d3-98ce-2fb7cdc62c83","Type":"ContainerDied","Data":"800e3754c05a8c98e820a8941972ec61258caf781254539f1d5a9d348a00828b"}
Jul 23 18:22:53.888553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.888543    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"
Jul 23 18:22:53.888772 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.888576    2582 scope.go:117] "RemoveContainer" containerID="64702ad444e68494ce7a7e64814fc5e8a135e441e3ff3a3a9b43b41a25173687"
Jul 23 18:22:53.914199 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.914165    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"]
Jul 23 18:22:53.917249 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.917221    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-090448/test-model-090448-kserve-655b6ccdf5-bhlwz"]
Jul 23 18:22:53.945647 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:53.945572    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:22:54.041905 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.041863    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-model-cache\") pod \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") "
Jul 23 18:22:54.041905 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.041908    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-home\") pod \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") "
Jul 23 18:22:54.042166 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.041929    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-dshm\") pod \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") "
Jul 23 18:22:54.042166 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.041955    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z7x8\" (UniqueName: \"kubernetes.io/projected/cb9b386e-e4ed-4633-a69a-15ed116a48f6-kube-api-access-6z7x8\") pod \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") "
Jul 23 18:22:54.042166 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.041983    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tmp-dir\") pod \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") "
Jul 23 18:22:54.042166 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.042013    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tls-certs\") pod \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\" (UID: \"cb9b386e-e4ed-4633-a69a-15ed116a48f6\") "
Jul 23 18:22:54.042166 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.042156    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-home" (OuterVolumeSpecName: "home") pod "cb9b386e-e4ed-4633-a69a-15ed116a48f6" (UID: "cb9b386e-e4ed-4633-a69a-15ed116a48f6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:54.042408 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.042182    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-model-cache" (OuterVolumeSpecName: "model-cache") pod "cb9b386e-e4ed-4633-a69a-15ed116a48f6" (UID: "cb9b386e-e4ed-4633-a69a-15ed116a48f6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:54.042408 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.042270    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "cb9b386e-e4ed-4633-a69a-15ed116a48f6" (UID: "cb9b386e-e4ed-4633-a69a-15ed116a48f6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:54.042408 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.042335    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:54.042408 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.042348    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:54.042408 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.042356    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:54.044222 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.044187    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-dshm" (OuterVolumeSpecName: "dshm") pod "cb9b386e-e4ed-4633-a69a-15ed116a48f6" (UID: "cb9b386e-e4ed-4633-a69a-15ed116a48f6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:22:54.044335 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.044233    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "cb9b386e-e4ed-4633-a69a-15ed116a48f6" (UID: "cb9b386e-e4ed-4633-a69a-15ed116a48f6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:22:54.044335 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.044238    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb9b386e-e4ed-4633-a69a-15ed116a48f6-kube-api-access-6z7x8" (OuterVolumeSpecName: "kube-api-access-6z7x8") pod "cb9b386e-e4ed-4633-a69a-15ed116a48f6" (UID: "cb9b386e-e4ed-4633-a69a-15ed116a48f6"). InnerVolumeSpecName "kube-api-access-6z7x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:22:54.143416 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.143371    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cb9b386e-e4ed-4633-a69a-15ed116a48f6-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:54.143416 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.143409    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z7x8\" (UniqueName: \"kubernetes.io/projected/cb9b386e-e4ed-4633-a69a-15ed116a48f6-kube-api-access-6z7x8\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:54.143416 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.143424    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cb9b386e-e4ed-4633-a69a-15ed116a48f6-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:22:54.896076 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.896042    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm" event={"ID":"cb9b386e-e4ed-4633-a69a-15ed116a48f6","Type":"ContainerDied","Data":"4e5b5a10fe2bd8329b5c43eaebbbe61d70c6d1e9a4cd96176382467db964f5cc"}
Jul 23 18:22:54.896076 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.896068    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"
Jul 23 18:22:54.896623 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.896090    2582 scope.go:117] "RemoveContainer" containerID="f7ad6396f1bf6a406021b0906ed53b3d2cdce40a8d0517ecb92f9b8b3c0afbfc"
Jul 23 18:22:54.920616 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.920581    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"]
Jul 23 18:22:54.935146 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:54.935108    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7893d7/test-model-7893d7-kserve-5955f985-97jdm"]
Jul 23 18:22:55.783935 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:55.783896    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="354815f9-26d1-43d3-98ce-2fb7cdc62c83" path="/var/lib/kubelet/pods/354815f9-26d1-43d3-98ce-2fb7cdc62c83/volumes"
Jul 23 18:22:55.784373 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:55.784353    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb9b386e-e4ed-4633-a69a-15ed116a48f6" path="/var/lib/kubelet/pods/cb9b386e-e4ed-4633-a69a-15ed116a48f6/volumes"
Jul 23 18:22:58.318437 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:22:58.318400    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:23:14.827750 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:23:14.827693    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:23:32.485202 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:23:32.485158    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:23:45.795582 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:23:45.795539    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:24:05.608540 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:24:05.608451    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:24:35.524333 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:24:35.524295    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:24:53.896624 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:24:53.896586    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:25:04.945330 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:04.945296    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:25:12.407549 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.407507    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"]
Jul 23 18:25:12.407987 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.407878    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb9b386e-e4ed-4633-a69a-15ed116a48f6" containerName="main"
Jul 23 18:25:12.407987 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.407894    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb9b386e-e4ed-4633-a69a-15ed116a48f6" containerName="main"
Jul 23 18:25:12.407987 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.407922    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="354815f9-26d1-43d3-98ce-2fb7cdc62c83" containerName="main"
Jul 23 18:25:12.407987 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.407928    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="354815f9-26d1-43d3-98ce-2fb7cdc62c83" containerName="main"
Jul 23 18:25:12.407987 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.407977    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb9b386e-e4ed-4633-a69a-15ed116a48f6" containerName="main"
Jul 23 18:25:12.407987 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.407988    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="354815f9-26d1-43d3-98ce-2fb7cdc62c83" containerName="main"
Jul 23 18:25:12.410943 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.410926    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.414048 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.414023    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"openshift-service-ca.crt\""
Jul 23 18:25:12.414156 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.414141    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"kube-root-ca.crt\""
Jul 23 18:25:12.415277 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.415256    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"default-dockercfg-5459x\""
Jul 23 18:25:12.415358 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.415277    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"auth-test-model-4d95fb-kserve-self-signed-certs\""
Jul 23 18:25:12.424024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.423995    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"]
Jul 23 18:25:12.497922 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.497883    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-dshm\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.497922 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.497918    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-tmp-dir\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.498148 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.497982    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db719d56-e60c-4092-bc75-e70c64bd8001-tls-certs\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.498148 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.498022    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-home\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.498148 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.498104    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-model-cache\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.498148 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.498124    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5zs6\" (UniqueName: \"kubernetes.io/projected/db719d56-e60c-4092-bc75-e70c64bd8001-kube-api-access-w5zs6\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599250 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599209    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-model-cache\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599250 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599249    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5zs6\" (UniqueName: \"kubernetes.io/projected/db719d56-e60c-4092-bc75-e70c64bd8001-kube-api-access-w5zs6\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599472 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599433    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-dshm\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599472 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599451    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-tmp-dir\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599554 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599494    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db719d56-e60c-4092-bc75-e70c64bd8001-tls-certs\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599554 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-home\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599701 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599670    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-model-cache\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599808 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599781    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-tmp-dir\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.599893 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.599873    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-home\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.601905 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.601877    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-dshm\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.602123 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.602106    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db719d56-e60c-4092-bc75-e70c64bd8001-tls-certs\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.608911 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.608880    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5zs6\" (UniqueName: \"kubernetes.io/projected/db719d56-e60c-4092-bc75-e70c64bd8001-kube-api-access-w5zs6\") pod \"auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") " pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.720468 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.720383    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:12.860815 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:12.860777    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"]
Jul 23 18:25:12.864622 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:25:12.864593    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb719d56_e60c_4092_bc75_e70c64bd8001.slice/crio-cffd885671ef72f01d3f492dddcc5e0acc8c7861bf206aa0693cfe9e23660dec WatchSource:0}: Error finding container cffd885671ef72f01d3f492dddcc5e0acc8c7861bf206aa0693cfe9e23660dec: Status 404 returned error can't find the container with id cffd885671ef72f01d3f492dddcc5e0acc8c7861bf206aa0693cfe9e23660dec
Jul 23 18:25:13.492033 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:13.491997    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2" event={"ID":"db719d56-e60c-4092-bc75-e70c64bd8001","Type":"ContainerStarted","Data":"3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6"}
Jul 23 18:25:13.492407 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:13.492043    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2" event={"ID":"db719d56-e60c-4092-bc75-e70c64bd8001","Type":"ContainerStarted","Data":"cffd885671ef72f01d3f492dddcc5e0acc8c7861bf206aa0693cfe9e23660dec"}
Jul 23 18:25:13.512800 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:13.512740    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2" podStartSLOduration=1.5127238219999999 podStartE2EDuration="1.512723822s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:25:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:25:13.511615391 +0000 UTC m=+2408.351123339" watchObservedRunningTime="2026-07-23 18:25:13.512723822 +0000 UTC m=+2408.352231760"
Jul 23 18:25:16.691227 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:16.691189    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:25:22.721427 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:22.721385    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:22.721857 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:22.721462    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:22.734047 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:22.734016    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:23.545323 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:23.545292    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:25:31.802725 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.802619    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"]
Jul 23 18:25:31.806350 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.806329    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.808963 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.808938    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"openshift-service-ca.crt\""
Jul 23 18:25:31.808963 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.808949    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"default-dockercfg-9m8gf\""
Jul 23 18:25:31.810067 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.810042    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"kube-root-ca.crt\""
Jul 23 18:25:31.810067 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.810056    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"auth-test-model-e1d86b-kserve-self-signed-certs\""
Jul 23 18:25:31.820111 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.820083    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"]
Jul 23 18:25:31.877303 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.877259    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-home\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.877482 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.877324    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fms9c\" (UniqueName: \"kubernetes.io/projected/a029b514-d87b-45d3-a6ba-2cf536071a69-kube-api-access-fms9c\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.877482 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.877395    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-dshm\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.877482 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.877434    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-tmp-dir\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.877482 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.877464    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-model-cache\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.877621 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.877515    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a029b514-d87b-45d3-a6ba-2cf536071a69-tls-certs\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978440 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978386    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-home\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978440 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978452    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fms9c\" (UniqueName: \"kubernetes.io/projected/a029b514-d87b-45d3-a6ba-2cf536071a69-kube-api-access-fms9c\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978757 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978488    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-dshm\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978757 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978512    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-tmp-dir\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978757 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978532    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-model-cache\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978757 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978564    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a029b514-d87b-45d3-a6ba-2cf536071a69-tls-certs\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978968 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978881    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-home\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.978968 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.978949    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-tmp-dir\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.979089 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.979000    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-model-cache\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.980921 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.980898    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-dshm\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.981032 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.981016    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a029b514-d87b-45d3-a6ba-2cf536071a69-tls-certs\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:31.988281 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:31.988226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fms9c\" (UniqueName: \"kubernetes.io/projected/a029b514-d87b-45d3-a6ba-2cf536071a69-kube-api-access-fms9c\") pod \"auth-test-model-e1d86b-kserve-665d7c648c-wlwh8\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") " pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:32.116065 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:32.115965    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:32.257774 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:32.257743    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"]
Jul 23 18:25:32.260122 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:25:32.260095    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda029b514_d87b_45d3_a6ba_2cf536071a69.slice/crio-736619c6852f1951714b9b9e278ac6a6eb04d04b4786a70a9f168cf431ca33dd WatchSource:0}: Error finding container 736619c6852f1951714b9b9e278ac6a6eb04d04b4786a70a9f168cf431ca33dd: Status 404 returned error can't find the container with id 736619c6852f1951714b9b9e278ac6a6eb04d04b4786a70a9f168cf431ca33dd
Jul 23 18:25:32.577270 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:32.577232    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8" event={"ID":"a029b514-d87b-45d3-a6ba-2cf536071a69","Type":"ContainerStarted","Data":"bca9852c30e3f4f7c5311e7f684f99725b2e168843741b8a1007666fd83d731a"}
Jul 23 18:25:32.577462 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:32.577278    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8" event={"ID":"a029b514-d87b-45d3-a6ba-2cf536071a69","Type":"ContainerStarted","Data":"736619c6852f1951714b9b9e278ac6a6eb04d04b4786a70a9f168cf431ca33dd"}
Jul 23 18:25:32.610694 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:32.610624    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8" podStartSLOduration=1.610602014 podStartE2EDuration="1.610602014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:25:32.608314104 +0000 UTC m=+2427.447822054" watchObservedRunningTime="2026-07-23 18:25:32.610602014 +0000 UTC m=+2427.450109968"
Jul 23 18:25:35.604380 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:35.604339    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:25:42.116660 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:42.116624    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:42.117107 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:42.116735    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:42.129008 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:42.128980    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:42.638658 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:42.638628    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:25:50.988483 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:25:50.988442    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:26:02.028246 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:26:02.028201    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:26:21.803665 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:26:21.803629    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:26:41.790766 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:26:41.790733    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:26:55.674669 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:26:55.674635    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:27:13.386032 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:13.385937    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:27:30.289834 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:30.289797    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:27:45.368677 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:45.368643    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"]
Jul 23 18:27:47.074509 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:47.074458    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"]
Jul 23 18:27:49.181673 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:49.181632    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:27:58.387381 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.387347    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"]
Jul 23 18:27:58.390594 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.390576    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.393055 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.393031    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"sub-test-model-4d95fb-kserve-self-signed-certs\""
Jul 23 18:27:58.404968 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.404940    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"]
Jul 23 18:27:58.442829 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.442790    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tmp-dir\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.443013 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.442848    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-model-cache\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.443013 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.442876    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tls-certs\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.443013 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.442950    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2ckt\" (UniqueName: \"kubernetes.io/projected/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-kube-api-access-n2ckt\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.443131 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.443031    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-dshm\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.443131 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.443064    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-home\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544177 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544142    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tmp-dir\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544381 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544192    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-model-cache\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544381 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544213    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tls-certs\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544381 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544248    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2ckt\" (UniqueName: \"kubernetes.io/projected/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-kube-api-access-n2ckt\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544381 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544294    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-dshm\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544381 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544337    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-home\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544650 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544603    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tmp-dir\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544768 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544749    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-home\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.544840 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.544793    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-model-cache\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.547098 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.547053    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tls-certs\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.547237 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.547130    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-dshm\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.553018 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.552996    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2ckt\" (UniqueName: \"kubernetes.io/projected/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-kube-api-access-n2ckt\") pod \"sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") " pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.701983 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.701886    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:27:58.837961 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:58.837928    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"]
Jul 23 18:27:58.840995 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:27:58.840965    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf9dd7c6a_d3b5_4e24_bb3f_b5e0a233549e.slice/crio-bcade780e9f4c05ef21087a7e2c2f402be877d9d6ea7f4d46e27675f7df8dc17 WatchSource:0}: Error finding container bcade780e9f4c05ef21087a7e2c2f402be877d9d6ea7f4d46e27675f7df8dc17: Status 404 returned error can't find the container with id bcade780e9f4c05ef21087a7e2c2f402be877d9d6ea7f4d46e27675f7df8dc17
Jul 23 18:27:59.246667 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:59.246628    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj" event={"ID":"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e","Type":"ContainerStarted","Data":"81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7"}
Jul 23 18:27:59.246667 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:59.246670    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj" event={"ID":"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e","Type":"ContainerStarted","Data":"bcade780e9f4c05ef21087a7e2c2f402be877d9d6ea7f4d46e27675f7df8dc17"}
Jul 23 18:27:59.269265 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:27:59.269211    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj" podStartSLOduration=1.269192379 podStartE2EDuration="1.269192379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:27:59.266932599 +0000 UTC m=+2574.106440567" watchObservedRunningTime="2026-07-23 18:27:59.269192379 +0000 UTC m=+2574.108700327"
Jul 23 18:28:00.388236 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.388188    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2" podUID="db719d56-e60c-4092-bc75-e70c64bd8001" containerName="main" containerID="cri-o://3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6" gracePeriod=45
Jul 23 18:28:00.646756 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.646663    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:28:00.765639 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.765601    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-model-cache\") pod \"db719d56-e60c-4092-bc75-e70c64bd8001\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") "
Jul 23 18:28:00.765639 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.765643    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db719d56-e60c-4092-bc75-e70c64bd8001-tls-certs\") pod \"db719d56-e60c-4092-bc75-e70c64bd8001\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") "
Jul 23 18:28:00.765927 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.765687    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-home\") pod \"db719d56-e60c-4092-bc75-e70c64bd8001\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") "
Jul 23 18:28:00.765927 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.765749    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-dshm\") pod \"db719d56-e60c-4092-bc75-e70c64bd8001\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") "
Jul 23 18:28:00.765927 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.765778    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5zs6\" (UniqueName: \"kubernetes.io/projected/db719d56-e60c-4092-bc75-e70c64bd8001-kube-api-access-w5zs6\") pod \"db719d56-e60c-4092-bc75-e70c64bd8001\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") "
Jul 23 18:28:00.765927 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.765891    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-tmp-dir\") pod \"db719d56-e60c-4092-bc75-e70c64bd8001\" (UID: \"db719d56-e60c-4092-bc75-e70c64bd8001\") "
Jul 23 18:28:00.766123 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.765948    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-model-cache" (OuterVolumeSpecName: "model-cache") pod "db719d56-e60c-4092-bc75-e70c64bd8001" (UID: "db719d56-e60c-4092-bc75-e70c64bd8001"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:00.766123 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.766042    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-home" (OuterVolumeSpecName: "home") pod "db719d56-e60c-4092-bc75-e70c64bd8001" (UID: "db719d56-e60c-4092-bc75-e70c64bd8001"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:00.766235 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.766130    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "db719d56-e60c-4092-bc75-e70c64bd8001" (UID: "db719d56-e60c-4092-bc75-e70c64bd8001"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:00.766281 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.766237    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:00.766281 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.766256    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:00.766281 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.766271    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:00.768100 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.768077    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-dshm" (OuterVolumeSpecName: "dshm") pod "db719d56-e60c-4092-bc75-e70c64bd8001" (UID: "db719d56-e60c-4092-bc75-e70c64bd8001"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:00.768209 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.768102    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db719d56-e60c-4092-bc75-e70c64bd8001-kube-api-access-w5zs6" (OuterVolumeSpecName: "kube-api-access-w5zs6") pod "db719d56-e60c-4092-bc75-e70c64bd8001" (UID: "db719d56-e60c-4092-bc75-e70c64bd8001"). InnerVolumeSpecName "kube-api-access-w5zs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:28:00.768209 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.768125    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/db719d56-e60c-4092-bc75-e70c64bd8001-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "db719d56-e60c-4092-bc75-e70c64bd8001" (UID: "db719d56-e60c-4092-bc75-e70c64bd8001"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:28:00.867761 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.867698    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db719d56-e60c-4092-bc75-e70c64bd8001-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:00.867761 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.867754    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5zs6\" (UniqueName: \"kubernetes.io/projected/db719d56-e60c-4092-bc75-e70c64bd8001-kube-api-access-w5zs6\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:00.867761 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:00.867765    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db719d56-e60c-4092-bc75-e70c64bd8001-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:01.261630 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.258394    2582 generic.go:356] "Generic (PLEG): container finished" podID="db719d56-e60c-4092-bc75-e70c64bd8001" containerID="3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6" exitCode=0
Jul 23 18:28:01.261630 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.258508    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2" event={"ID":"db719d56-e60c-4092-bc75-e70c64bd8001","Type":"ContainerDied","Data":"3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6"}
Jul 23 18:28:01.261630 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.258553    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2" event={"ID":"db719d56-e60c-4092-bc75-e70c64bd8001","Type":"ContainerDied","Data":"cffd885671ef72f01d3f492dddcc5e0acc8c7861bf206aa0693cfe9e23660dec"}
Jul 23 18:28:01.261630 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.258577    2582 scope.go:117] "RemoveContainer" containerID="3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6"
Jul 23 18:28:01.261630 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.258810    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"
Jul 23 18:28:01.269313 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.269285    2582 scope.go:117] "RemoveContainer" containerID="3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6"
Jul 23 18:28:01.269619 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:28:01.269598    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6\": container with ID starting with 3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6 not found: ID does not exist" containerID="3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6"
Jul 23 18:28:01.269679 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.269628    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6"} err="failed to get container status \"3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6\": rpc error: code = NotFound desc = could not find container \"3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6\": container with ID starting with 3b748968fdc77f6da57739b0ee15f2b095a2743cda5be06310816d4a7c99acc6 not found: ID does not exist"
Jul 23 18:28:01.284069 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.284026    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"]
Jul 23 18:28:01.290076 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.290048    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/auth-test-model-4d95fb-kserve-7d5556c5d7-wf8z2"]
Jul 23 18:28:01.784903 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:01.784871    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db719d56-e60c-4092-bc75-e70c64bd8001" path="/var/lib/kubelet/pods/db719d56-e60c-4092-bc75-e70c64bd8001/volumes"
Jul 23 18:28:02.094427 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.094385    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8" podUID="a029b514-d87b-45d3-a6ba-2cf536071a69" containerName="main" containerID="cri-o://bca9852c30e3f4f7c5311e7f684f99725b2e168843741b8a1007666fd83d731a" gracePeriod=45
Jul 23 18:28:02.266581 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.266136    2582 generic.go:356] "Generic (PLEG): container finished" podID="a029b514-d87b-45d3-a6ba-2cf536071a69" containerID="bca9852c30e3f4f7c5311e7f684f99725b2e168843741b8a1007666fd83d731a" exitCode=0
Jul 23 18:28:02.266581 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.266254    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8" event={"ID":"a029b514-d87b-45d3-a6ba-2cf536071a69","Type":"ContainerDied","Data":"bca9852c30e3f4f7c5311e7f684f99725b2e168843741b8a1007666fd83d731a"}
Jul 23 18:28:02.343045 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.343017    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:28:02.482404 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482318    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-model-cache\") pod \"a029b514-d87b-45d3-a6ba-2cf536071a69\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") "
Jul 23 18:28:02.482404 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482371    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-home\") pod \"a029b514-d87b-45d3-a6ba-2cf536071a69\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") "
Jul 23 18:28:02.482404 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482399    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a029b514-d87b-45d3-a6ba-2cf536071a69-tls-certs\") pod \"a029b514-d87b-45d3-a6ba-2cf536071a69\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") "
Jul 23 18:28:02.482686 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482436    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-dshm\") pod \"a029b514-d87b-45d3-a6ba-2cf536071a69\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") "
Jul 23 18:28:02.482686 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482457    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fms9c\" (UniqueName: \"kubernetes.io/projected/a029b514-d87b-45d3-a6ba-2cf536071a69-kube-api-access-fms9c\") pod \"a029b514-d87b-45d3-a6ba-2cf536071a69\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") "
Jul 23 18:28:02.482686 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482565    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-tmp-dir\") pod \"a029b514-d87b-45d3-a6ba-2cf536071a69\" (UID: \"a029b514-d87b-45d3-a6ba-2cf536071a69\") "
Jul 23 18:28:02.482686 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482605    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-model-cache" (OuterVolumeSpecName: "model-cache") pod "a029b514-d87b-45d3-a6ba-2cf536071a69" (UID: "a029b514-d87b-45d3-a6ba-2cf536071a69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:02.482686 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482638    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-home" (OuterVolumeSpecName: "home") pod "a029b514-d87b-45d3-a6ba-2cf536071a69" (UID: "a029b514-d87b-45d3-a6ba-2cf536071a69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:02.482957 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482909    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a029b514-d87b-45d3-a6ba-2cf536071a69" (UID: "a029b514-d87b-45d3-a6ba-2cf536071a69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:02.482957 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482947    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:02.483060 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.482964    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:02.484810 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.484766    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a029b514-d87b-45d3-a6ba-2cf536071a69-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a029b514-d87b-45d3-a6ba-2cf536071a69" (UID: "a029b514-d87b-45d3-a6ba-2cf536071a69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:28:02.485040 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.485016    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-dshm" (OuterVolumeSpecName: "dshm") pod "a029b514-d87b-45d3-a6ba-2cf536071a69" (UID: "a029b514-d87b-45d3-a6ba-2cf536071a69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:28:02.485144 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.485032    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a029b514-d87b-45d3-a6ba-2cf536071a69-kube-api-access-fms9c" (OuterVolumeSpecName: "kube-api-access-fms9c") pod "a029b514-d87b-45d3-a6ba-2cf536071a69" (UID: "a029b514-d87b-45d3-a6ba-2cf536071a69"). InnerVolumeSpecName "kube-api-access-fms9c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:28:02.584321 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.584279    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a029b514-d87b-45d3-a6ba-2cf536071a69-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:02.584321 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.584313    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:02.584321 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.584322    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fms9c\" (UniqueName: \"kubernetes.io/projected/a029b514-d87b-45d3-a6ba-2cf536071a69-kube-api-access-fms9c\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:02.584571 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.584334    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a029b514-d87b-45d3-a6ba-2cf536071a69-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:28:02.900005 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:02.899974    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:28:03.273429 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:03.273346    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"
Jul 23 18:28:03.273601 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:03.273346    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8" event={"ID":"a029b514-d87b-45d3-a6ba-2cf536071a69","Type":"ContainerDied","Data":"736619c6852f1951714b9b9e278ac6a6eb04d04b4786a70a9f168cf431ca33dd"}
Jul 23 18:28:03.273601 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:03.273470    2582 scope.go:117] "RemoveContainer" containerID="bca9852c30e3f4f7c5311e7f684f99725b2e168843741b8a1007666fd83d731a"
Jul 23 18:28:03.294150 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:03.294120    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"]
Jul 23 18:28:03.298159 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:03.298132    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/auth-test-model-e1d86b-kserve-665d7c648c-wlwh8"]
Jul 23 18:28:03.784171 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:03.784138    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a029b514-d87b-45d3-a6ba-2cf536071a69" path="/var/lib/kubelet/pods/a029b514-d87b-45d3-a6ba-2cf536071a69/volumes"
Jul 23 18:28:08.702679 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:08.702641    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:28:08.703101 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:08.702698    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:28:08.715600 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:08.715573    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:28:09.313040 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:09.313012    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:28:17.771668 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.771628    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"]
Jul 23 18:28:17.774096 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.772043    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a029b514-d87b-45d3-a6ba-2cf536071a69" containerName="main"
Jul 23 18:28:17.774096 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.772057    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="a029b514-d87b-45d3-a6ba-2cf536071a69" containerName="main"
Jul 23 18:28:17.774096 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.772075    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db719d56-e60c-4092-bc75-e70c64bd8001" containerName="main"
Jul 23 18:28:17.774096 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.772081    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="db719d56-e60c-4092-bc75-e70c64bd8001" containerName="main"
Jul 23 18:28:17.774096 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.772139    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="a029b514-d87b-45d3-a6ba-2cf536071a69" containerName="main"
Jul 23 18:28:17.774096 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.772149    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="db719d56-e60c-4092-bc75-e70c64bd8001" containerName="main"
Jul 23 18:28:17.774929 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.774905    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:17.777403 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.777381    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"openshift-service-ca.crt\""
Jul 23 18:28:17.777726 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.777684    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"kube-root-ca.crt\""
Jul 23 18:28:17.778480 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.778454    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"sub-test-model-e1d86b-kserve-self-signed-certs\""
Jul 23 18:28:17.779407 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.778970    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"default-dockercfg-9m8gf\""
Jul 23 18:28:17.786244 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.786220    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"]
Jul 23 18:28:17.933803 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.933759    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tls-certs\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:17.934024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.933872    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-dshm\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:17.934024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.933905    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-model-cache\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:17.934024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.933932    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tmp-dir\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:17.934024 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.933961    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcwkf\" (UniqueName: \"kubernetes.io/projected/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-kube-api-access-bcwkf\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:17.934176 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:17.934030    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-home\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035279 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035179    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-dshm\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035279 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035221    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-model-cache\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035279 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035240    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tmp-dir\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035279 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035258    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcwkf\" (UniqueName: \"kubernetes.io/projected/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-kube-api-access-bcwkf\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035279 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035285    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-home\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035690 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035325    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tls-certs\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035793 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035747    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tmp-dir\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035793 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035775    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-home\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.035885 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.035798    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-model-cache\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.037442 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.037423    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-dshm\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.037721 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.037689    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tls-certs\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.044977 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.044957    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcwkf\" (UniqueName: \"kubernetes.io/projected/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-kube-api-access-bcwkf\") pod \"sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") " pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.086062 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.086019    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:18.220671 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.220642    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"]
Jul 23 18:28:18.225448 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:28:18.225124    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb24f51e4_de5a_450f_90a0_7a3a5ea2723b.slice/crio-c3fd4b516693c24ecd8dcf55d5eb0329b5395ffa9e71196253bc97058bfa3f7b WatchSource:0}: Error finding container c3fd4b516693c24ecd8dcf55d5eb0329b5395ffa9e71196253bc97058bfa3f7b: Status 404 returned error can't find the container with id c3fd4b516693c24ecd8dcf55d5eb0329b5395ffa9e71196253bc97058bfa3f7b
Jul 23 18:28:18.346277 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.346236    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4" event={"ID":"b24f51e4-de5a-450f-90a0-7a3a5ea2723b","Type":"ContainerStarted","Data":"1a35fcd6e7f7e218ce2911a089ed2767891f3e01e275632c71cb96113de61a8e"}
Jul 23 18:28:18.346277 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.346293    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4" event={"ID":"b24f51e4-de5a-450f-90a0-7a3a5ea2723b","Type":"ContainerStarted","Data":"c3fd4b516693c24ecd8dcf55d5eb0329b5395ffa9e71196253bc97058bfa3f7b"}
Jul 23 18:28:18.365571 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:18.365518    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4" podStartSLOduration=1.365503012 podStartE2EDuration="1.365503012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:28:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:28:18.364312997 +0000 UTC m=+2593.203820946" watchObservedRunningTime="2026-07-23 18:28:18.365503012 +0000 UTC m=+2593.205011004"
Jul 23 18:28:21.587445 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:21.587404    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:28:28.087205 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:28.087118    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:28.087205 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:28.087171    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:28.099822 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:28.099794    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:28.402171 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:28.402083    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:28:36.025004 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:36.024965    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:28:56.992019 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:28:56.991977    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:29:03.515113 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:03.515077    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"]
Jul 23 18:29:04.467824 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:04.467788    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"]
Jul 23 18:29:16.190577 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:16.190544    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:29:18.534876 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.534836    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj" podUID="f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" containerName="main" containerID="cri-o://81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7" gracePeriod=45
Jul 23 18:29:18.785879 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.785807    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:29:18.901969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.901932    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tmp-dir\") pod \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") "
Jul 23 18:29:18.901969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.901978    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2ckt\" (UniqueName: \"kubernetes.io/projected/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-kube-api-access-n2ckt\") pod \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") "
Jul 23 18:29:18.902208 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902016    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-model-cache\") pod \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") "
Jul 23 18:29:18.902208 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902076    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-home\") pod \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") "
Jul 23 18:29:18.902208 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902143    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tls-certs\") pod \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") "
Jul 23 18:29:18.902208 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902200    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-dshm\") pod \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\" (UID: \"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e\") "
Jul 23 18:29:18.902420 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902255    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-model-cache" (OuterVolumeSpecName: "model-cache") pod "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" (UID: "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:18.902420 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902270    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" (UID: "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:18.902510 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902414    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-home" (OuterVolumeSpecName: "home") pod "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" (UID: "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:18.902510 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902440    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:18.902510 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.902454    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:18.904247 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.904221    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-dshm" (OuterVolumeSpecName: "dshm") pod "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" (UID: "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:18.904370 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.904289    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" (UID: "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:29:18.904370 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:18.904292    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-kube-api-access-n2ckt" (OuterVolumeSpecName: "kube-api-access-n2ckt") pod "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" (UID: "f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e"). InnerVolumeSpecName "kube-api-access-n2ckt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:29:19.003143 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.003099    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.003143 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.003136    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.003143 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.003150    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.003395 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.003163    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2ckt\" (UniqueName: \"kubernetes.io/projected/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e-kube-api-access-n2ckt\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.487732 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.487643    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4" podUID="b24f51e4-de5a-450f-90a0-7a3a5ea2723b" containerName="main" containerID="cri-o://1a35fcd6e7f7e218ce2911a089ed2767891f3e01e275632c71cb96113de61a8e" gracePeriod=45
Jul 23 18:29:19.614462 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.614429    2582 generic.go:356] "Generic (PLEG): container finished" podID="f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" containerID="81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7" exitCode=0
Jul 23 18:29:19.614944 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.614477    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"
Jul 23 18:29:19.614944 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.614503    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj" event={"ID":"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e","Type":"ContainerDied","Data":"81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7"}
Jul 23 18:29:19.614944 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.614543    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj" event={"ID":"f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e","Type":"ContainerDied","Data":"bcade780e9f4c05ef21087a7e2c2f402be877d9d6ea7f4d46e27675f7df8dc17"}
Jul 23 18:29:19.614944 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.614564    2582 scope.go:117] "RemoveContainer" containerID="81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7"
Jul 23 18:29:19.616448 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.616426    2582 generic.go:356] "Generic (PLEG): container finished" podID="b24f51e4-de5a-450f-90a0-7a3a5ea2723b" containerID="1a35fcd6e7f7e218ce2911a089ed2767891f3e01e275632c71cb96113de61a8e" exitCode=0
Jul 23 18:29:19.616564 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.616491    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4" event={"ID":"b24f51e4-de5a-450f-90a0-7a3a5ea2723b","Type":"ContainerDied","Data":"1a35fcd6e7f7e218ce2911a089ed2767891f3e01e275632c71cb96113de61a8e"}
Jul 23 18:29:19.624274 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.624250    2582 scope.go:117] "RemoveContainer" containerID="81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7"
Jul 23 18:29:19.624640 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:29:19.624618    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7\": container with ID starting with 81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7 not found: ID does not exist" containerID="81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7"
Jul 23 18:29:19.624766 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.624648    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7"} err="failed to get container status \"81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7\": rpc error: code = NotFound desc = could not find container \"81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7\": container with ID starting with 81204f03a8da0339ea49807a4a1dcf802358c023504287aaaa77ceef16f9b2d7 not found: ID does not exist"
Jul 23 18:29:19.643111 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.643077    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"]
Jul 23 18:29:19.645519 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.645497    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/sub-test-model-4d95fb-kserve-548d76d5ff-zsvcj"]
Jul 23 18:29:19.743898 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.743815    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:29:19.794011 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.793966    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" path="/var/lib/kubelet/pods/f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e/volumes"
Jul 23 18:29:19.811074 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811033    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-model-cache\") pod \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") "
Jul 23 18:29:19.811269 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811100    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-home\") pod \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") "
Jul 23 18:29:19.811269 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811140    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-dshm\") pod \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") "
Jul 23 18:29:19.811269 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811171    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tmp-dir\") pod \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") "
Jul 23 18:29:19.811269 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811197    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcwkf\" (UniqueName: \"kubernetes.io/projected/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-kube-api-access-bcwkf\") pod \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") "
Jul 23 18:29:19.811269 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811258    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tls-certs\") pod \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\" (UID: \"b24f51e4-de5a-450f-90a0-7a3a5ea2723b\") "
Jul 23 18:29:19.811510 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811327    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-model-cache" (OuterVolumeSpecName: "model-cache") pod "b24f51e4-de5a-450f-90a0-7a3a5ea2723b" (UID: "b24f51e4-de5a-450f-90a0-7a3a5ea2723b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:19.811510 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811410    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b24f51e4-de5a-450f-90a0-7a3a5ea2723b" (UID: "b24f51e4-de5a-450f-90a0-7a3a5ea2723b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:19.811510 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811421    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-home" (OuterVolumeSpecName: "home") pod "b24f51e4-de5a-450f-90a0-7a3a5ea2723b" (UID: "b24f51e4-de5a-450f-90a0-7a3a5ea2723b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:19.811654 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811566    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.811654 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811580    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.811654 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.811589    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.813442 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.813412    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-kube-api-access-bcwkf" (OuterVolumeSpecName: "kube-api-access-bcwkf") pod "b24f51e4-de5a-450f-90a0-7a3a5ea2723b" (UID: "b24f51e4-de5a-450f-90a0-7a3a5ea2723b"). InnerVolumeSpecName "kube-api-access-bcwkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:29:19.813921 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.813881    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-dshm" (OuterVolumeSpecName: "dshm") pod "b24f51e4-de5a-450f-90a0-7a3a5ea2723b" (UID: "b24f51e4-de5a-450f-90a0-7a3a5ea2723b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:29:19.814046 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.814022    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b24f51e4-de5a-450f-90a0-7a3a5ea2723b" (UID: "b24f51e4-de5a-450f-90a0-7a3a5ea2723b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:29:19.912903 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.912856    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.912903 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.912900    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:19.913125 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:19.912913    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcwkf\" (UniqueName: \"kubernetes.io/projected/b24f51e4-de5a-450f-90a0-7a3a5ea2723b-kube-api-access-bcwkf\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:29:20.622476 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:20.622446    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"
Jul 23 18:29:20.622476 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:20.622457    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4" event={"ID":"b24f51e4-de5a-450f-90a0-7a3a5ea2723b","Type":"ContainerDied","Data":"c3fd4b516693c24ecd8dcf55d5eb0329b5395ffa9e71196253bc97058bfa3f7b"}
Jul 23 18:29:20.623056 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:20.622507    2582 scope.go:117] "RemoveContainer" containerID="1a35fcd6e7f7e218ce2911a089ed2767891f3e01e275632c71cb96113de61a8e"
Jul 23 18:29:20.643443 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:20.643412    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"]
Jul 23 18:29:20.647147 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:20.647120    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/sub-test-model-e1d86b-kserve-5bbc89c6f9-wd7s4"]
Jul 23 18:29:21.784028 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:21.783991    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b24f51e4-de5a-450f-90a0-7a3a5ea2723b" path="/var/lib/kubelet/pods/b24f51e4-de5a-450f-90a0-7a3a5ea2723b/volumes"
Jul 23 18:29:22.482543 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.482509    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"]
Jul 23 18:29:22.482866 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.482853    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b24f51e4-de5a-450f-90a0-7a3a5ea2723b" containerName="main"
Jul 23 18:29:22.482923 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.482868    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b24f51e4-de5a-450f-90a0-7a3a5ea2723b" containerName="main"
Jul 23 18:29:22.482923 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.482890    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" containerName="main"
Jul 23 18:29:22.482923 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.482896    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" containerName="main"
Jul 23 18:29:22.483015 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.482955    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b24f51e4-de5a-450f-90a0-7a3a5ea2723b" containerName="main"
Jul 23 18:29:22.483015 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.482966    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9dd7c6a-d3b5-4e24-bb3f-b5e0a233549e" containerName="main"
Jul 23 18:29:22.487258 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.487241    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.489959 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.489939    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"default-dockercfg-5459x\""
Jul 23 18:29:22.490948 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.490926    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"kube-root-ca.crt\""
Jul 23 18:29:22.491066 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.491051    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"rate-test-model-4d95fb-kserve-self-signed-certs\""
Jul 23 18:29:22.491192 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.491178    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-4d95fb\"/\"openshift-service-ca.crt\""
Jul 23 18:29:22.500190 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.500167    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"]
Jul 23 18:29:22.539097 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.539056    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-tmp-dir\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.539097 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.539099    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-home\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.539314 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.539117    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-model-cache\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.539314 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.539156    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d4e0da-51f2-44d7-9824-207729719c63-tls-certs\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.539314 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.539241    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-dshm\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.539314 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.539260    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pdprn\" (UniqueName: \"kubernetes.io/projected/89d4e0da-51f2-44d7-9824-207729719c63-kube-api-access-pdprn\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.640572 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.640533    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-tmp-dir\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.640572 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.640579    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-home\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.640872 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.640598    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-model-cache\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.640872 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.640622    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d4e0da-51f2-44d7-9824-207729719c63-tls-certs\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.640872 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.640644    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-dshm\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.640872 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.640766    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pdprn\" (UniqueName: \"kubernetes.io/projected/89d4e0da-51f2-44d7-9824-207729719c63-kube-api-access-pdprn\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.641075 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.641044    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-tmp-dir\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.641134 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.641071    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-model-cache\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.641402 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.641378    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-home\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.643273 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.643250    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-dshm\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.643528 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.643509    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d4e0da-51f2-44d7-9824-207729719c63-tls-certs\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.648645 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.648620    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pdprn\" (UniqueName: \"kubernetes.io/projected/89d4e0da-51f2-44d7-9824-207729719c63-kube-api-access-pdprn\") pod \"rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") " pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.797263 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.797168    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:22.942835 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:22.942805    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"]
Jul 23 18:29:22.944931 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:29:22.944902    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod89d4e0da_51f2_44d7_9824_207729719c63.slice/crio-aaba1d62d92aa84f9b626ec0430de903c382d948caffa034ef6cbb5b824ec614 WatchSource:0}: Error finding container aaba1d62d92aa84f9b626ec0430de903c382d948caffa034ef6cbb5b824ec614: Status 404 returned error can't find the container with id aaba1d62d92aa84f9b626ec0430de903c382d948caffa034ef6cbb5b824ec614
Jul 23 18:29:23.640637 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:23.640600    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp" event={"ID":"89d4e0da-51f2-44d7-9824-207729719c63","Type":"ContainerStarted","Data":"4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f"}
Jul 23 18:29:23.640637 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:23.640641    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp" event={"ID":"89d4e0da-51f2-44d7-9824-207729719c63","Type":"ContainerStarted","Data":"aaba1d62d92aa84f9b626ec0430de903c382d948caffa034ef6cbb5b824ec614"}
Jul 23 18:29:23.662576 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:23.662518    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp" podStartSLOduration=1.662500037 podStartE2EDuration="1.662500037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:29:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:29:23.66145393 +0000 UTC m=+2658.500961880" watchObservedRunningTime="2026-07-23 18:29:23.662500037 +0000 UTC m=+2658.502007984"
Jul 23 18:29:25.978690 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:25.978650    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:29:32.797342 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:32.797306    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:32.797769 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:32.797369    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:32.810336 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:32.810302    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:33.704550 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:33.704508    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:29:44.985333 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:44.985288    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"]
Jul 23 18:29:44.990313 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:44.990281    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:44.992897 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:44.992869    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"openshift-service-ca.crt\""
Jul 23 18:29:44.993053 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:44.992915    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"kube-root-ca.crt\""
Jul 23 18:29:44.994058 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:44.994035    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"default-dockercfg-9m8gf\""
Jul 23 18:29:44.994354 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:44.994043    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-e1d86b\"/\"rate-test-model-e1d86b-kserve-self-signed-certs\""
Jul 23 18:29:44.996296 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:44.996276    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"]
Jul 23 18:29:45.144766 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.144729    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5lcc\" (UniqueName: \"kubernetes.io/projected/f04c8811-f9fe-4521-9710-cfed5f189ce6-kube-api-access-b5lcc\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.144766 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.144769    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-model-cache\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.145048 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.144835    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-home\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.145048 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.144913    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-tmp-dir\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.145048 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.144970    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-dshm\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.145048 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.144993    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f04c8811-f9fe-4521-9710-cfed5f189ce6-tls-certs\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.246426 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5lcc\" (UniqueName: \"kubernetes.io/projected/f04c8811-f9fe-4521-9710-cfed5f189ce6-kube-api-access-b5lcc\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.246426 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246377    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-model-cache\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.246426 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246422    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-home\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.246750 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246460    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-tmp-dir\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.246750 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246500    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-dshm\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.246750 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246521    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f04c8811-f9fe-4521-9710-cfed5f189ce6-tls-certs\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.246951 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246925    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-home\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.247011 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246958    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-tmp-dir\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.247011 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.246999    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-model-cache\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.248888 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.248869    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-dshm\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.249139 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.249119    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f04c8811-f9fe-4521-9710-cfed5f189ce6-tls-certs\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.254482 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.254457    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5lcc\" (UniqueName: \"kubernetes.io/projected/f04c8811-f9fe-4521-9710-cfed5f189ce6-kube-api-access-b5lcc\") pod \"rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") " pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.302313 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.302272    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:45.437969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.437944    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"]
Jul 23 18:29:45.440472 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:29:45.440446    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf04c8811_f9fe_4521_9710_cfed5f189ce6.slice/crio-f2aaf2a2821811b4bcd77d715d37b87aae4214edf8357b93fcd28dbc503df76f WatchSource:0}: Error finding container f2aaf2a2821811b4bcd77d715d37b87aae4214edf8357b93fcd28dbc503df76f: Status 404 returned error can't find the container with id f2aaf2a2821811b4bcd77d715d37b87aae4214edf8357b93fcd28dbc503df76f
Jul 23 18:29:45.749762 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.749721    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw" event={"ID":"f04c8811-f9fe-4521-9710-cfed5f189ce6","Type":"ContainerStarted","Data":"af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52"}
Jul 23 18:29:45.749762 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.749764    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw" event={"ID":"f04c8811-f9fe-4521-9710-cfed5f189ce6","Type":"ContainerStarted","Data":"f2aaf2a2821811b4bcd77d715d37b87aae4214edf8357b93fcd28dbc503df76f"}
Jul 23 18:29:45.768824 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:45.768767    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw" podStartSLOduration=1.76874852 podStartE2EDuration="1.76874852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:29:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:29:45.767221555 +0000 UTC m=+2680.606729494" watchObservedRunningTime="2026-07-23 18:29:45.76874852 +0000 UTC m=+2680.608256468"
Jul 23 18:29:48.686047 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:48.686007    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:29:55.302980 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:55.302938    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:55.303426 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:55.303251    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:55.315840 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:55.315811    2582 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:29:55.806471 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:29:55.806449    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:30:03.700685 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:03.700594    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:30:18.879194 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:18.879131    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:30:26.105320 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:26.105285    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"]
Jul 23 18:30:26.995756 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:26.995721    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"]
Jul 23 18:30:31.221929 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.221893    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7415-140008656604720"]
Jul 23 18:30:31.225377 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.225359    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:31.227946 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.227924    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-r99xg\""
Jul 23 18:30:31.235086 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.235065    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656604720"]
Jul 23 18:30:31.271330 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.271287    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjpp2\" (UniqueName: \"kubernetes.io/projected/05916649-77b7-4548-bf83-b2a7aeacde35-kube-api-access-mjpp2\") pod \"test-curl-7415-140008656604720\" (UID: \"05916649-77b7-4548-bf83-b2a7aeacde35\") " pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:31.372650 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.372611    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjpp2\" (UniqueName: \"kubernetes.io/projected/05916649-77b7-4548-bf83-b2a7aeacde35-kube-api-access-mjpp2\") pod \"test-curl-7415-140008656604720\" (UID: \"05916649-77b7-4548-bf83-b2a7aeacde35\") " pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:31.383091 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.383064    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjpp2\" (UniqueName: \"kubernetes.io/projected/05916649-77b7-4548-bf83-b2a7aeacde35-kube-api-access-mjpp2\") pod \"test-curl-7415-140008656604720\" (UID: \"05916649-77b7-4548-bf83-b2a7aeacde35\") " pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:31.535615 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.535496    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:31.711378 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.711353    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656604720"]
Jul 23 18:30:31.713021 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:30:31.712998    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod05916649_77b7_4548_bf83_b2a7aeacde35.slice/crio-61f0a58d0579146673d7b4ac531230631b2b7c5b234fc43308221890d1f6f19e WatchSource:0}: Error finding container 61f0a58d0579146673d7b4ac531230631b2b7c5b234fc43308221890d1f6f19e: Status 404 returned error can't find the container with id 61f0a58d0579146673d7b4ac531230631b2b7c5b234fc43308221890d1f6f19e
Jul 23 18:30:31.714310 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.714289    2582 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 18:30:31.959724 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:31.959676    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656604720" event={"ID":"05916649-77b7-4548-bf83-b2a7aeacde35","Type":"ContainerStarted","Data":"61f0a58d0579146673d7b4ac531230631b2b7c5b234fc43308221890d1f6f19e"}
Jul 23 18:30:32.966258 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:32.966221    2582 generic.go:356] "Generic (PLEG): container finished" podID="05916649-77b7-4548-bf83-b2a7aeacde35" containerID="ac2d6060ee981ddc2ac23b68c2ebe8838c3b67963c72a6113df4f4e16a937e37" exitCode=0
Jul 23 18:30:32.966648 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:32.966308    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656604720" event={"ID":"05916649-77b7-4548-bf83-b2a7aeacde35","Type":"ContainerDied","Data":"ac2d6060ee981ddc2ac23b68c2ebe8838c3b67963c72a6113df4f4e16a937e37"}
Jul 23 18:30:34.105419 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.105386    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:34.112348 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.112322    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:30:34.148949 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.148919    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7415-140008656604720_05916649-77b7-4548-bf83-b2a7aeacde35/test-curl-7415-140008656604720/0.log"
Jul 23 18:30:34.175908 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.175872    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7415-140008656604720"]
Jul 23 18:30:34.183657 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.183623    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7415-140008656604720"]
Jul 23 18:30:34.197909 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.197883    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mjpp2\" (UniqueName: \"kubernetes.io/projected/05916649-77b7-4548-bf83-b2a7aeacde35-kube-api-access-mjpp2\") pod \"05916649-77b7-4548-bf83-b2a7aeacde35\" (UID: \"05916649-77b7-4548-bf83-b2a7aeacde35\") "
Jul 23 18:30:34.200083 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.200057    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05916649-77b7-4548-bf83-b2a7aeacde35-kube-api-access-mjpp2" (OuterVolumeSpecName: "kube-api-access-mjpp2") pod "05916649-77b7-4548-bf83-b2a7aeacde35" (UID: "05916649-77b7-4548-bf83-b2a7aeacde35"). InnerVolumeSpecName "kube-api-access-mjpp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:30:34.298896 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.298815    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mjpp2\" (UniqueName: \"kubernetes.io/projected/05916649-77b7-4548-bf83-b2a7aeacde35-kube-api-access-mjpp2\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:34.346850 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.346813    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:34.347176 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.347162    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05916649-77b7-4548-bf83-b2a7aeacde35" containerName="test-curl-7415-140008656604720"
Jul 23 18:30:34.347219 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.347179    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="05916649-77b7-4548-bf83-b2a7aeacde35" containerName="test-curl-7415-140008656604720"
Jul 23 18:30:34.347254 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.347231    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="05916649-77b7-4548-bf83-b2a7aeacde35" containerName="test-curl-7415-140008656604720"
Jul 23 18:30:34.350211 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.350196    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:34.358261 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.358236    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:34.399757 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.399691    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvnzc\" (UniqueName: \"kubernetes.io/projected/d84b09c1-075b-4c69-b56e-3cbcf50caa13-kube-api-access-cvnzc\") pod \"test-curl-7415-140008656598624\" (UID: \"d84b09c1-075b-4c69-b56e-3cbcf50caa13\") " pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:34.500477 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.500432    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvnzc\" (UniqueName: \"kubernetes.io/projected/d84b09c1-075b-4c69-b56e-3cbcf50caa13-kube-api-access-cvnzc\") pod \"test-curl-7415-140008656598624\" (UID: \"d84b09c1-075b-4c69-b56e-3cbcf50caa13\") " pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:34.509460 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.509436    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvnzc\" (UniqueName: \"kubernetes.io/projected/d84b09c1-075b-4c69-b56e-3cbcf50caa13-kube-api-access-cvnzc\") pod \"test-curl-7415-140008656598624\" (UID: \"d84b09c1-075b-4c69-b56e-3cbcf50caa13\") " pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:34.660431 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.660391    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:34.812594 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.812558    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:34.817500 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:30:34.817461    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd84b09c1_075b_4c69_b56e_3cbcf50caa13.slice/crio-9a0dcc04a5cd44b555d82d938d9deed4ae75770b23a3c6af06265782b0d3a892 WatchSource:0}: Error finding container 9a0dcc04a5cd44b555d82d938d9deed4ae75770b23a3c6af06265782b0d3a892: Status 404 returned error can't find the container with id 9a0dcc04a5cd44b555d82d938d9deed4ae75770b23a3c6af06265782b0d3a892
Jul 23 18:30:34.976971 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.976883    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61f0a58d0579146673d7b4ac531230631b2b7c5b234fc43308221890d1f6f19e"
Jul 23 18:30:34.976971 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.976882    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:34.978201 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:34.978170    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656598624" event={"ID":"d84b09c1-075b-4c69-b56e-3cbcf50caa13","Type":"ContainerStarted","Data":"9a0dcc04a5cd44b555d82d938d9deed4ae75770b23a3c6af06265782b0d3a892"}
Jul 23 18:30:34.990891 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:34.990852    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7415-140008656604720\" is forbidden: User \"system:node:ip-10-0-139-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-151.ec2.internal' and this object" podUID="05916649-77b7-4548-bf83-b2a7aeacde35" pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:35.782946 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:35.782901    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7415-140008656604720\" is forbidden: User \"system:node:ip-10-0-139-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-151.ec2.internal' and this object" podUID="05916649-77b7-4548-bf83-b2a7aeacde35" pod="opendatahub/test-curl-7415-140008656604720"
Jul 23 18:30:35.784160 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:35.784138    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05916649-77b7-4548-bf83-b2a7aeacde35" path="/var/lib/kubelet/pods/05916649-77b7-4548-bf83-b2a7aeacde35/volumes"
Jul 23 18:30:35.984513 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:35.984470    2582 generic.go:356] "Generic (PLEG): container finished" podID="d84b09c1-075b-4c69-b56e-3cbcf50caa13" containerID="6b959eb37b65846971669f73901c0e78c357a0579b19855c73532dea9611b293" exitCode=0
Jul 23 18:30:35.984734 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:35.984524    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656598624" event={"ID":"d84b09c1-075b-4c69-b56e-3cbcf50caa13","Type":"ContainerDied","Data":"6b959eb37b65846971669f73901c0e78c357a0579b19855c73532dea9611b293"}
Jul 23 18:30:37.125410 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.125377    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:37.150576 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.150550    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7415-140008656598624_d84b09c1-075b-4c69-b56e-3cbcf50caa13/test-curl-7415-140008656598624/0.log"
Jul 23 18:30:37.178569 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.178528    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:37.185284 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.185252    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:37.227386 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.227347    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvnzc\" (UniqueName: \"kubernetes.io/projected/d84b09c1-075b-4c69-b56e-3cbcf50caa13-kube-api-access-cvnzc\") pod \"d84b09c1-075b-4c69-b56e-3cbcf50caa13\" (UID: \"d84b09c1-075b-4c69-b56e-3cbcf50caa13\") "
Jul 23 18:30:37.229494 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.229465    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d84b09c1-075b-4c69-b56e-3cbcf50caa13-kube-api-access-cvnzc" (OuterVolumeSpecName: "kube-api-access-cvnzc") pod "d84b09c1-075b-4c69-b56e-3cbcf50caa13" (UID: "d84b09c1-075b-4c69-b56e-3cbcf50caa13"). InnerVolumeSpecName "kube-api-access-cvnzc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:30:37.305025 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.304988    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:37.305332 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.305319    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d84b09c1-075b-4c69-b56e-3cbcf50caa13" containerName="test-curl-7415-140008656598624"
Jul 23 18:30:37.305387 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.305334    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="d84b09c1-075b-4c69-b56e-3cbcf50caa13" containerName="test-curl-7415-140008656598624"
Jul 23 18:30:37.305420 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.305400    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="d84b09c1-075b-4c69-b56e-3cbcf50caa13" containerName="test-curl-7415-140008656598624"
Jul 23 18:30:37.308560 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.308543    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:37.316089 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.316063    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:37.328248 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.328217    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cl4x\" (UniqueName: \"kubernetes.io/projected/b9bf9951-453c-46d7-84e2-41f75407fff9-kube-api-access-4cl4x\") pod \"test-curl-7415-140008656598624\" (UID: \"b9bf9951-453c-46d7-84e2-41f75407fff9\") " pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:37.328441 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.328416    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvnzc\" (UniqueName: \"kubernetes.io/projected/d84b09c1-075b-4c69-b56e-3cbcf50caa13-kube-api-access-cvnzc\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:37.429472 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.429420    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cl4x\" (UniqueName: \"kubernetes.io/projected/b9bf9951-453c-46d7-84e2-41f75407fff9-kube-api-access-4cl4x\") pod \"test-curl-7415-140008656598624\" (UID: \"b9bf9951-453c-46d7-84e2-41f75407fff9\") " pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:37.438261 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.438226    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cl4x\" (UniqueName: \"kubernetes.io/projected/b9bf9951-453c-46d7-84e2-41f75407fff9-kube-api-access-4cl4x\") pod \"test-curl-7415-140008656598624\" (UID: \"b9bf9951-453c-46d7-84e2-41f75407fff9\") " pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:37.618754 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.618621    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:37.768194 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.768160    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:37.771242 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:30:37.771207    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9bf9951_453c_46d7_84e2_41f75407fff9.slice/crio-116ba2842b835d58ea94aa6752d161eb7f61bba1e91618d844f0c510e247f01f WatchSource:0}: Error finding container 116ba2842b835d58ea94aa6752d161eb7f61bba1e91618d844f0c510e247f01f: Status 404 returned error can't find the container with id 116ba2842b835d58ea94aa6752d161eb7f61bba1e91618d844f0c510e247f01f
Jul 23 18:30:37.783799 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.783774    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d84b09c1-075b-4c69-b56e-3cbcf50caa13" path="/var/lib/kubelet/pods/d84b09c1-075b-4c69-b56e-3cbcf50caa13/volumes"
Jul 23 18:30:37.995180 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.995091    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:37.995180 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.995140    2582 scope.go:117] "RemoveContainer" containerID="6b959eb37b65846971669f73901c0e78c357a0579b19855c73532dea9611b293"
Jul 23 18:30:37.996621 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:37.996428    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656598624" event={"ID":"b9bf9951-453c-46d7-84e2-41f75407fff9","Type":"ContainerStarted","Data":"116ba2842b835d58ea94aa6752d161eb7f61bba1e91618d844f0c510e247f01f"}
Jul 23 18:30:39.003290 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:39.003254    2582 generic.go:356] "Generic (PLEG): container finished" podID="b9bf9951-453c-46d7-84e2-41f75407fff9" containerID="6e904f1c0bdc6fd7d3accde74a88aeb3ea78d842732cae32189564a9b9a40563" exitCode=0
Jul 23 18:30:39.003673 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:39.003309    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656598624" event={"ID":"b9bf9951-453c-46d7-84e2-41f75407fff9","Type":"ContainerDied","Data":"6e904f1c0bdc6fd7d3accde74a88aeb3ea78d842732cae32189564a9b9a40563"}
Jul 23 18:30:40.144351 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:40.144327    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:40.167424 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:40.167394    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7415-140008656598624_b9bf9951-453c-46d7-84e2-41f75407fff9/test-curl-7415-140008656598624/0.log"
Jul 23 18:30:40.195334 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:40.195292    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:40.200467 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:40.200439    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7415-140008656598624"]
Jul 23 18:30:40.257415 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:40.257384    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cl4x\" (UniqueName: \"kubernetes.io/projected/b9bf9951-453c-46d7-84e2-41f75407fff9-kube-api-access-4cl4x\") pod \"b9bf9951-453c-46d7-84e2-41f75407fff9\" (UID: \"b9bf9951-453c-46d7-84e2-41f75407fff9\") "
Jul 23 18:30:40.259578 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:40.259521    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b9bf9951-453c-46d7-84e2-41f75407fff9-kube-api-access-4cl4x" (OuterVolumeSpecName: "kube-api-access-4cl4x") pod "b9bf9951-453c-46d7-84e2-41f75407fff9" (UID: "b9bf9951-453c-46d7-84e2-41f75407fff9"). InnerVolumeSpecName "kube-api-access-4cl4x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:30:40.358421 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:40.358383    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cl4x\" (UniqueName: \"kubernetes.io/projected/b9bf9951-453c-46d7-84e2-41f75407fff9-kube-api-access-4cl4x\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:41.014224 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.014194    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:41.014403 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.014193    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="116ba2842b835d58ea94aa6752d161eb7f61bba1e91618d844f0c510e247f01f"
Jul 23 18:30:41.024051 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:41.024019    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7415-140008656598624\" is forbidden: User \"system:node:ip-10-0-139-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-151.ec2.internal' and this object" podUID="b9bf9951-453c-46d7-84e2-41f75407fff9" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:41.125355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.125307    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp" podUID="89d4e0da-51f2-44d7-9824-207729719c63" containerName="main" containerID="cri-o://4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f" gracePeriod=45
Jul 23 18:30:41.379458 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.379426    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:30:41.381885 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:41.381858    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7415-140008656598624\" is forbidden: User \"system:node:ip-10-0-139-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-151.ec2.internal' and this object" podUID="b9bf9951-453c-46d7-84e2-41f75407fff9" pod="opendatahub/test-curl-7415-140008656598624"
Jul 23 18:30:41.471770 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.471701    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pdprn\" (UniqueName: \"kubernetes.io/projected/89d4e0da-51f2-44d7-9824-207729719c63-kube-api-access-pdprn\") pod \"89d4e0da-51f2-44d7-9824-207729719c63\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") "
Jul 23 18:30:41.471945 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.471787    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-home\") pod \"89d4e0da-51f2-44d7-9824-207729719c63\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") "
Jul 23 18:30:41.471945 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.471819    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-tmp-dir\") pod \"89d4e0da-51f2-44d7-9824-207729719c63\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") "
Jul 23 18:30:41.471945 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.471849    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-model-cache\") pod \"89d4e0da-51f2-44d7-9824-207729719c63\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") "
Jul 23 18:30:41.471945 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.471870    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d4e0da-51f2-44d7-9824-207729719c63-tls-certs\") pod \"89d4e0da-51f2-44d7-9824-207729719c63\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") "
Jul 23 18:30:41.472167 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.471952    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-dshm\") pod \"89d4e0da-51f2-44d7-9824-207729719c63\" (UID: \"89d4e0da-51f2-44d7-9824-207729719c63\") "
Jul 23 18:30:41.472167 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.472052    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-home" (OuterVolumeSpecName: "home") pod "89d4e0da-51f2-44d7-9824-207729719c63" (UID: "89d4e0da-51f2-44d7-9824-207729719c63"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:41.472167 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.472086    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-model-cache" (OuterVolumeSpecName: "model-cache") pod "89d4e0da-51f2-44d7-9824-207729719c63" (UID: "89d4e0da-51f2-44d7-9824-207729719c63"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:41.472167 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.472102    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "89d4e0da-51f2-44d7-9824-207729719c63" (UID: "89d4e0da-51f2-44d7-9824-207729719c63"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:41.472361 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.472262    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:41.472361 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.472273    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:41.472361 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.472284    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:41.474016 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.473994    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89d4e0da-51f2-44d7-9824-207729719c63-kube-api-access-pdprn" (OuterVolumeSpecName: "kube-api-access-pdprn") pod "89d4e0da-51f2-44d7-9824-207729719c63" (UID: "89d4e0da-51f2-44d7-9824-207729719c63"). InnerVolumeSpecName "kube-api-access-pdprn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:30:41.474125 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.474071    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-dshm" (OuterVolumeSpecName: "dshm") pod "89d4e0da-51f2-44d7-9824-207729719c63" (UID: "89d4e0da-51f2-44d7-9824-207729719c63"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:41.474167 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.474124    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89d4e0da-51f2-44d7-9824-207729719c63-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "89d4e0da-51f2-44d7-9824-207729719c63" (UID: "89d4e0da-51f2-44d7-9824-207729719c63"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:30:41.573128 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.573085    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/89d4e0da-51f2-44d7-9824-207729719c63-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:41.573128 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.573120    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pdprn\" (UniqueName: \"kubernetes.io/projected/89d4e0da-51f2-44d7-9824-207729719c63-kube-api-access-pdprn\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:41.573128 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.573131    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/89d4e0da-51f2-44d7-9824-207729719c63-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:41.785035 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:41.784996    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b9bf9951-453c-46d7-84e2-41f75407fff9" path="/var/lib/kubelet/pods/b9bf9951-453c-46d7-84e2-41f75407fff9/volumes"
Jul 23 18:30:42.016054 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.015936    2582 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw" podUID="f04c8811-f9fe-4521-9710-cfed5f189ce6" containerName="main" containerID="cri-o://af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52" gracePeriod=45
Jul 23 18:30:42.020520 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.020491    2582 generic.go:356] "Generic (PLEG): container finished" podID="89d4e0da-51f2-44d7-9824-207729719c63" containerID="4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f" exitCode=0
Jul 23 18:30:42.020660 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.020532    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"
Jul 23 18:30:42.020660 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.020570    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp" event={"ID":"89d4e0da-51f2-44d7-9824-207729719c63","Type":"ContainerDied","Data":"4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f"}
Jul 23 18:30:42.020660 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.020634    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp" event={"ID":"89d4e0da-51f2-44d7-9824-207729719c63","Type":"ContainerDied","Data":"aaba1d62d92aa84f9b626ec0430de903c382d948caffa034ef6cbb5b824ec614"}
Jul 23 18:30:42.020660 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.020656    2582 scope.go:117] "RemoveContainer" containerID="4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f"
Jul 23 18:30:42.061889 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.061854    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"]
Jul 23 18:30:42.071195 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.071168    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-4d95fb/rate-test-model-4d95fb-kserve-78ccb79d8c-2hpfp"]
Jul 23 18:30:42.079272 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.079251    2582 scope.go:117] "RemoveContainer" containerID="4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f"
Jul 23 18:30:42.079805 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:42.079781    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f\": container with ID starting with 4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f not found: ID does not exist" containerID="4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f"
Jul 23 18:30:42.079882 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.079816    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f"} err="failed to get container status \"4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f\": rpc error: code = NotFound desc = could not find container \"4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f\": container with ID starting with 4c3c04358750bf16700c1a34948f494c36b66ac16e085469cc1ed7c1bd073f3f not found: ID does not exist"
Jul 23 18:30:42.266482 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.266423    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:30:42.381383 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381344    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5lcc\" (UniqueName: \"kubernetes.io/projected/f04c8811-f9fe-4521-9710-cfed5f189ce6-kube-api-access-b5lcc\") pod \"f04c8811-f9fe-4521-9710-cfed5f189ce6\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") "
Jul 23 18:30:42.381383 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381388    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-model-cache\") pod \"f04c8811-f9fe-4521-9710-cfed5f189ce6\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") "
Jul 23 18:30:42.381969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381410    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-home\") pod \"f04c8811-f9fe-4521-9710-cfed5f189ce6\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") "
Jul 23 18:30:42.381969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381437    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-dshm\") pod \"f04c8811-f9fe-4521-9710-cfed5f189ce6\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") "
Jul 23 18:30:42.381969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381563    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f04c8811-f9fe-4521-9710-cfed5f189ce6-tls-certs\") pod \"f04c8811-f9fe-4521-9710-cfed5f189ce6\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") "
Jul 23 18:30:42.381969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381632    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-tmp-dir\") pod \"f04c8811-f9fe-4521-9710-cfed5f189ce6\" (UID: \"f04c8811-f9fe-4521-9710-cfed5f189ce6\") "
Jul 23 18:30:42.381969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381700    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-model-cache" (OuterVolumeSpecName: "model-cache") pod "f04c8811-f9fe-4521-9710-cfed5f189ce6" (UID: "f04c8811-f9fe-4521-9710-cfed5f189ce6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:42.381969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381770    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-home" (OuterVolumeSpecName: "home") pod "f04c8811-f9fe-4521-9710-cfed5f189ce6" (UID: "f04c8811-f9fe-4521-9710-cfed5f189ce6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:42.381969 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.381925    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f04c8811-f9fe-4521-9710-cfed5f189ce6" (UID: "f04c8811-f9fe-4521-9710-cfed5f189ce6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:42.382220 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.382048    2582 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-model-cache\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:42.382220 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.382067    2582 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-home\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:42.382220 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.382080    2582 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-tmp-dir\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:42.383672 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.383642    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-dshm" (OuterVolumeSpecName: "dshm") pod "f04c8811-f9fe-4521-9710-cfed5f189ce6" (UID: "f04c8811-f9fe-4521-9710-cfed5f189ce6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 18:30:42.383797 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.383687    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f04c8811-f9fe-4521-9710-cfed5f189ce6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f04c8811-f9fe-4521-9710-cfed5f189ce6" (UID: "f04c8811-f9fe-4521-9710-cfed5f189ce6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 18:30:42.383797 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.383751    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f04c8811-f9fe-4521-9710-cfed5f189ce6-kube-api-access-b5lcc" (OuterVolumeSpecName: "kube-api-access-b5lcc") pod "f04c8811-f9fe-4521-9710-cfed5f189ce6" (UID: "f04c8811-f9fe-4521-9710-cfed5f189ce6"). InnerVolumeSpecName "kube-api-access-b5lcc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:30:42.483377 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.483333    2582 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f04c8811-f9fe-4521-9710-cfed5f189ce6-dshm\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:42.483377 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.483370    2582 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f04c8811-f9fe-4521-9710-cfed5f189ce6-tls-certs\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:42.483377 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:42.483386    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5lcc\" (UniqueName: \"kubernetes.io/projected/f04c8811-f9fe-4521-9710-cfed5f189ce6-kube-api-access-b5lcc\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:43.028360 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.028329    2582 generic.go:356] "Generic (PLEG): container finished" podID="f04c8811-f9fe-4521-9710-cfed5f189ce6" containerID="af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52" exitCode=0
Jul 23 18:30:43.028360 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.028362    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"
Jul 23 18:30:43.028360 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.028373    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw" event={"ID":"f04c8811-f9fe-4521-9710-cfed5f189ce6","Type":"ContainerDied","Data":"af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52"}
Jul 23 18:30:43.028678 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.028412    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw" event={"ID":"f04c8811-f9fe-4521-9710-cfed5f189ce6","Type":"ContainerDied","Data":"f2aaf2a2821811b4bcd77d715d37b87aae4214edf8357b93fcd28dbc503df76f"}
Jul 23 18:30:43.028678 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.028431    2582 scope.go:117] "RemoveContainer" containerID="af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52"
Jul 23 18:30:43.037679 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.037647    2582 scope.go:117] "RemoveContainer" containerID="af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52"
Jul 23 18:30:43.037939 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:43.037921    2582 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52\": container with ID starting with af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52 not found: ID does not exist" containerID="af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52"
Jul 23 18:30:43.038002 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.037946    2582 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52"} err="failed to get container status \"af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52\": rpc error: code = NotFound desc = could not find container \"af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52\": container with ID starting with af8576a7e6f9fe09ff1c6af55182d5fd8bea7505fe0455563b010fc32b429a52 not found: ID does not exist"
Jul 23 18:30:43.055230 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.055195    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"]
Jul 23 18:30:43.061396 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.061366    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-e1d86b/rate-test-model-e1d86b-kserve-7cfcbc4667-vr6zw"]
Jul 23 18:30:43.061927 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:43.061904    2582 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf04c8811_f9fe_4521_9710_cfed5f189ce6.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf04c8811_f9fe_4521_9710_cfed5f189ce6.slice/crio-f2aaf2a2821811b4bcd77d715d37b87aae4214edf8357b93fcd28dbc503df76f\": RecentStats: unable to find data in memory cache]"
Jul 23 18:30:43.242199 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242165    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7415-140008656553424"]
Jul 23 18:30:43.242509 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242495    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b9bf9951-453c-46d7-84e2-41f75407fff9" containerName="test-curl-7415-140008656598624"
Jul 23 18:30:43.242553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242511    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="b9bf9951-453c-46d7-84e2-41f75407fff9" containerName="test-curl-7415-140008656598624"
Jul 23 18:30:43.242553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242531    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f04c8811-f9fe-4521-9710-cfed5f189ce6" containerName="main"
Jul 23 18:30:43.242553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242537    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="f04c8811-f9fe-4521-9710-cfed5f189ce6" containerName="main"
Jul 23 18:30:43.242553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242545    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89d4e0da-51f2-44d7-9824-207729719c63" containerName="main"
Jul 23 18:30:43.242553 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242551    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="89d4e0da-51f2-44d7-9824-207729719c63" containerName="main"
Jul 23 18:30:43.242728 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242601    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="f04c8811-f9fe-4521-9710-cfed5f189ce6" containerName="main"
Jul 23 18:30:43.242728 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242608    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="b9bf9951-453c-46d7-84e2-41f75407fff9" containerName="test-curl-7415-140008656598624"
Jul 23 18:30:43.242728 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.242616    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="89d4e0da-51f2-44d7-9824-207729719c63" containerName="main"
Jul 23 18:30:43.245452 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.245435    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:43.247655 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.247635    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-r99xg\""
Jul 23 18:30:43.251248 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.251223    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656553424"]
Jul 23 18:30:43.289875 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.289785    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l75tf\" (UniqueName: \"kubernetes.io/projected/4d319898-b5aa-4667-8a96-3e006c992093-kube-api-access-l75tf\") pod \"test-curl-7415-140008656553424\" (UID: \"4d319898-b5aa-4667-8a96-3e006c992093\") " pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:43.390691 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.390645    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l75tf\" (UniqueName: \"kubernetes.io/projected/4d319898-b5aa-4667-8a96-3e006c992093-kube-api-access-l75tf\") pod \"test-curl-7415-140008656553424\" (UID: \"4d319898-b5aa-4667-8a96-3e006c992093\") " pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:43.398368 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.398336    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l75tf\" (UniqueName: \"kubernetes.io/projected/4d319898-b5aa-4667-8a96-3e006c992093-kube-api-access-l75tf\") pod \"test-curl-7415-140008656553424\" (UID: \"4d319898-b5aa-4667-8a96-3e006c992093\") " pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:43.560144 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.560112    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:43.697126 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.697098    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7415-140008656553424"]
Jul 23 18:30:43.699842 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:30:43.699814    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d319898_b5aa_4667_8a96_3e006c992093.slice/crio-bdcc20b249a2ed0f76f47f76fbbf784ee0e19ef1f63c58077c5309f4027ea067 WatchSource:0}: Error finding container bdcc20b249a2ed0f76f47f76fbbf784ee0e19ef1f63c58077c5309f4027ea067: Status 404 returned error can't find the container with id bdcc20b249a2ed0f76f47f76fbbf784ee0e19ef1f63c58077c5309f4027ea067
Jul 23 18:30:43.784410 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.784375    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89d4e0da-51f2-44d7-9824-207729719c63" path="/var/lib/kubelet/pods/89d4e0da-51f2-44d7-9824-207729719c63/volumes"
Jul 23 18:30:43.784837 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:43.784824    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f04c8811-f9fe-4521-9710-cfed5f189ce6" path="/var/lib/kubelet/pods/f04c8811-f9fe-4521-9710-cfed5f189ce6/volumes"
Jul 23 18:30:44.033765 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:44.033727    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656553424" event={"ID":"4d319898-b5aa-4667-8a96-3e006c992093","Type":"ContainerStarted","Data":"bdcc20b249a2ed0f76f47f76fbbf784ee0e19ef1f63c58077c5309f4027ea067"}
Jul 23 18:30:45.041183 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:45.041152    2582 generic.go:356] "Generic (PLEG): container finished" podID="4d319898-b5aa-4667-8a96-3e006c992093" containerID="a5f6a897e7075d2a39e746863045387e1dd390fd674bb7b440358f64d32ec035" exitCode=0
Jul 23 18:30:45.041584 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:45.041214    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7415-140008656553424" event={"ID":"4d319898-b5aa-4667-8a96-3e006c992093","Type":"ContainerDied","Data":"a5f6a897e7075d2a39e746863045387e1dd390fd674bb7b440358f64d32ec035"}
Jul 23 18:30:46.190200 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:46.190169    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:46.211723 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:46.211677    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7415-140008656553424_4d319898-b5aa-4667-8a96-3e006c992093/test-curl-7415-140008656553424/0.log"
Jul 23 18:30:46.240509 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:46.240472    2582 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7415-140008656553424"]
Jul 23 18:30:46.244099 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:46.244074    2582 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7415-140008656553424"]
Jul 23 18:30:46.318102 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:46.318075    2582 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l75tf\" (UniqueName: \"kubernetes.io/projected/4d319898-b5aa-4667-8a96-3e006c992093-kube-api-access-l75tf\") pod \"4d319898-b5aa-4667-8a96-3e006c992093\" (UID: \"4d319898-b5aa-4667-8a96-3e006c992093\") "
Jul 23 18:30:46.320319 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:46.320288    2582 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d319898-b5aa-4667-8a96-3e006c992093-kube-api-access-l75tf" (OuterVolumeSpecName: "kube-api-access-l75tf") pod "4d319898-b5aa-4667-8a96-3e006c992093" (UID: "4d319898-b5aa-4667-8a96-3e006c992093"). InnerVolumeSpecName "kube-api-access-l75tf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 18:30:46.418901 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:46.418864    2582 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l75tf\" (UniqueName: \"kubernetes.io/projected/4d319898-b5aa-4667-8a96-3e006c992093-kube-api-access-l75tf\") on node \"ip-10-0-139-151.ec2.internal\" DevicePath \"\""
Jul 23 18:30:47.052196 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:47.052165    2582 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:47.052379 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:47.052201    2582 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bdcc20b249a2ed0f76f47f76fbbf784ee0e19ef1f63c58077c5309f4027ea067"
Jul 23 18:30:47.062045 ip-10-0-139-151 kubenswrapper[2582]: E0723 18:30:47.062019    2582 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7415-140008656553424\" is forbidden: User \"system:node:ip-10-0-139-151.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-139-151.ec2.internal' and this object" podUID="4d319898-b5aa-4667-8a96-3e006c992093" pod="opendatahub/test-curl-7415-140008656553424"
Jul 23 18:30:47.788639 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:30:47.788598    2582 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d319898-b5aa-4667-8a96-3e006c992093" path="/var/lib/kubelet/pods/4d319898-b5aa-4667-8a96-3e006c992093/volumes"
Jul 23 18:31:02.634028 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:02.633986    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:31:22.385323 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:22.385277    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-x5svp"]
Jul 23 18:31:31.878281 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:31.878191    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54bdbf49cd-p4dpg_911b1f50-9941-4abb-8411-85987a0ffb0a/authorino/0.log"
Jul 23 18:31:37.651077 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:37.651049    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9xks5_a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785/server/0.log"
Jul 23 18:31:38.018757 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:38.018642    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547c58d5d9-hbh8z_5dcac08f-5065-4dfa-bd0a-d5ccba92c976/manager/0.log"
Jul 23 18:31:38.029813 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:38.029792    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547c58d5d9-hbh8z_5dcac08f-5065-4dfa-bd0a-d5ccba92c976/copy-manifests/0.log"
Jul 23 18:31:40.247394 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:40.247346    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54bdbf49cd-p4dpg_911b1f50-9941-4abb-8411-85987a0ffb0a/authorino/0.log"
Jul 23 18:31:40.597533 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:40.597506    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hcngm_5da948c6-e962-4e8a-9204-7f9212241836/kuadrant-console-plugin/0.log"
Jul 23 18:31:40.952081 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:40.951994    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-x5svp_02174c34-a9c8-45b4-829f-c79cc0adfdd3/limitador/0.log"
Jul 23 18:31:41.424274 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:41.424245    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h72j8_8fefef8d-63df-4f11-84a0-483681b242cb/istio-proxy/0.log"
Jul 23 18:31:41.659646 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:41.659614    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6c5d7974-k9nj7_f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2/kube-auth-proxy/0.log"
Jul 23 18:31:41.902784 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:41.902726    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5sn9_d2eb6cca-0902-4227-8a43-b223e53fd3b2/istio-proxy/1.log"
Jul 23 18:31:43.229153 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:43.229127    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-w9t8v_3ed1d7d3-98e0-4388-a52d-18a8da279e80/main/0.log"
Jul 23 18:31:48.477225 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.477189    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v5cbl/must-gather-wbwrj"]
Jul 23 18:31:48.477624 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.477504    2582 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d319898-b5aa-4667-8a96-3e006c992093" containerName="test-curl-7415-140008656553424"
Jul 23 18:31:48.477624 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.477514    2582 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d319898-b5aa-4667-8a96-3e006c992093" containerName="test-curl-7415-140008656553424"
Jul 23 18:31:48.477624 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.477577    2582 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d319898-b5aa-4667-8a96-3e006c992093" containerName="test-curl-7415-140008656553424"
Jul 23 18:31:48.480375 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.480360    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.487001 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.486978    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v5cbl\"/\"kube-root-ca.crt\""
Jul 23 18:31:48.487125 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.486999    2582 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-v5cbl\"/\"default-dockercfg-qc65b\""
Jul 23 18:31:48.487125 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.487114    2582 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-v5cbl\"/\"openshift-service-ca.crt\""
Jul 23 18:31:48.505925 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.505891    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v5cbl/must-gather-wbwrj"]
Jul 23 18:31:48.562267 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.562216    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9xs9\" (UniqueName: \"kubernetes.io/projected/6d98b856-3ff9-4026-b615-4e263dc0ea3b-kube-api-access-p9xs9\") pod \"must-gather-wbwrj\" (UID: \"6d98b856-3ff9-4026-b615-4e263dc0ea3b\") " pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.562267 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.562277    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6d98b856-3ff9-4026-b615-4e263dc0ea3b-must-gather-output\") pod \"must-gather-wbwrj\" (UID: \"6d98b856-3ff9-4026-b615-4e263dc0ea3b\") " pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.663419 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.663382    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9xs9\" (UniqueName: \"kubernetes.io/projected/6d98b856-3ff9-4026-b615-4e263dc0ea3b-kube-api-access-p9xs9\") pod \"must-gather-wbwrj\" (UID: \"6d98b856-3ff9-4026-b615-4e263dc0ea3b\") " pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.663600 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.663429    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6d98b856-3ff9-4026-b615-4e263dc0ea3b-must-gather-output\") pod \"must-gather-wbwrj\" (UID: \"6d98b856-3ff9-4026-b615-4e263dc0ea3b\") " pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.663787 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.663771    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6d98b856-3ff9-4026-b615-4e263dc0ea3b-must-gather-output\") pod \"must-gather-wbwrj\" (UID: \"6d98b856-3ff9-4026-b615-4e263dc0ea3b\") " pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.672272 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.672236    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9xs9\" (UniqueName: \"kubernetes.io/projected/6d98b856-3ff9-4026-b615-4e263dc0ea3b-kube-api-access-p9xs9\") pod \"must-gather-wbwrj\" (UID: \"6d98b856-3ff9-4026-b615-4e263dc0ea3b\") " pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.789626 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.789517    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v5cbl/must-gather-wbwrj"
Jul 23 18:31:48.928613 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:48.928587    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v5cbl/must-gather-wbwrj"]
Jul 23 18:31:48.931166 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:31:48.931139    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d98b856_3ff9_4026_b615_4e263dc0ea3b.slice/crio-e945c7f5908b7f5364f5fee0ee1105fd354e6fb0baa1d5cd4d612ea4b19fae50 WatchSource:0}: Error finding container e945c7f5908b7f5364f5fee0ee1105fd354e6fb0baa1d5cd4d612ea4b19fae50: Status 404 returned error can't find the container with id e945c7f5908b7f5364f5fee0ee1105fd354e6fb0baa1d5cd4d612ea4b19fae50
Jul 23 18:31:49.336071 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:49.336035    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v5cbl/must-gather-wbwrj" event={"ID":"6d98b856-3ff9-4026-b615-4e263dc0ea3b","Type":"ContainerStarted","Data":"e945c7f5908b7f5364f5fee0ee1105fd354e6fb0baa1d5cd4d612ea4b19fae50"}
Jul 23 18:31:50.344938 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:50.344833    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v5cbl/must-gather-wbwrj" event={"ID":"6d98b856-3ff9-4026-b615-4e263dc0ea3b","Type":"ContainerStarted","Data":"55e52f7f5ad4acc99498026b3d6f2d88450291b81563a36e1bc894a6bba134e1"}
Jul 23 18:31:50.345478 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:50.345449    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v5cbl/must-gather-wbwrj" event={"ID":"6d98b856-3ff9-4026-b615-4e263dc0ea3b","Type":"ContainerStarted","Data":"ecf49ce2a575dfc00a150b56e37f0cb1e98ac350c12f4bd5c4eaea5466f80cbf"}
Jul 23 18:31:51.716751 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:51.716695    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tcqbt_dde8d19a-bea3-48da-9fee-a5353c5e385e/global-pull-secret-syncer/0.log"
Jul 23 18:31:51.974891 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:51.974810    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xn4ct_ed8d602a-92db-4f5e-86a3-7f25c4209c6e/konnectivity-agent/0.log"
Jul 23 18:31:52.045640 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:52.045599    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-151.ec2.internal_845580dd794356674919cca719ca31b8/haproxy/0.log"
Jul 23 18:31:56.950734 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:56.950071    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-54bdbf49cd-p4dpg_911b1f50-9941-4abb-8411-85987a0ffb0a/authorino/0.log"
Jul 23 18:31:57.032732 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:57.032681    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-hcngm_5da948c6-e962-4e8a-9204-7f9212241836/kuadrant-console-plugin/0.log"
Jul 23 18:31:57.273372 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:57.273287    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-x5svp_02174c34-a9c8-45b4-829f-c79cc0adfdd3/limitador/0.log"
Jul 23 18:31:59.127345 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.127313    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-sllgk_d0ff0b11-a868-4bb3-a040-ece238d061d7/kube-state-metrics/0.log"
Jul 23 18:31:59.148612 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.148569    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-sllgk_d0ff0b11-a868-4bb3-a040-ece238d061d7/kube-rbac-proxy-main/0.log"
Jul 23 18:31:59.171596 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.171495    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-sllgk_d0ff0b11-a868-4bb3-a040-ece238d061d7/kube-rbac-proxy-self/0.log"
Jul 23 18:31:59.205101 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.205071    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-59bd5df7f6-vkwhg_7289462b-47e1-4b5f-9901-b6248a57f4ec/metrics-server/0.log"
Jul 23 18:31:59.246281 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.246245    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-ghkth_8d5d8fe0-d14b-43f3-8e5d-68d815ff9cc3/monitoring-plugin/0.log"
Jul 23 18:31:59.474152 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.474115    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrmgs_211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0/node-exporter/0.log"
Jul 23 18:31:59.501298 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.501261    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrmgs_211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0/kube-rbac-proxy/0.log"
Jul 23 18:31:59.523254 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.523216    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qrmgs_211e2e1a-fa7f-4d51-ab73-e1cf89b94dc0/init-textfile/0.log"
Jul 23 18:31:59.640492 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.640458    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_084afdb7-8a71-4b96-a774-9487ce0ffd9e/prometheus/0.log"
Jul 23 18:31:59.663425 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.663371    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_084afdb7-8a71-4b96-a774-9487ce0ffd9e/config-reloader/0.log"
Jul 23 18:31:59.694474 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.694449    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_084afdb7-8a71-4b96-a774-9487ce0ffd9e/thanos-sidecar/0.log"
Jul 23 18:31:59.719848 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.719766    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_084afdb7-8a71-4b96-a774-9487ce0ffd9e/kube-rbac-proxy-web/0.log"
Jul 23 18:31:59.744324 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.744296    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_084afdb7-8a71-4b96-a774-9487ce0ffd9e/kube-rbac-proxy/0.log"
Jul 23 18:31:59.765768 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.765741    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_084afdb7-8a71-4b96-a774-9487ce0ffd9e/kube-rbac-proxy-thanos/0.log"
Jul 23 18:31:59.787370 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.787344    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_084afdb7-8a71-4b96-a774-9487ce0ffd9e/init-config-reloader/0.log"
Jul 23 18:31:59.818986 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.818952    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-5gnmj_86ff8c6e-49f8-4662-8e95-56cabf77cde1/prometheus-operator/0.log"
Jul 23 18:31:59.837658 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:31:59.837625    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-5gnmj_86ff8c6e-49f8-4662-8e95-56cabf77cde1/kube-rbac-proxy/0.log"
Jul 23 18:32:02.193095 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.193027    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v5cbl/must-gather-wbwrj" podStartSLOduration=13.051816282 podStartE2EDuration="14.193002202s" totalImagesPullingTime="1.14118592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 18:31:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:31:50.361306196 +0000 UTC m=+2805.200814167" watchObservedRunningTime="2026-07-23 18:32:02.193002202 +0000 UTC m=+2817.032510176"
Jul 23 18:32:02.193754 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.193732    2582 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"]
Jul 23 18:32:02.199481 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.199440    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.219861 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.219827    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"]
Jul 23 18:32:02.310925 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.310882    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-proc\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.311137 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.310947    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-sys\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.311137 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.311002    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-podres\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.311137 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.311046    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-lib-modules\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.311137 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.311097    2582 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56mnd\" (UniqueName: \"kubernetes.io/projected/99d1aedb-1bac-4475-8b63-2370a104d55b-kube-api-access-56mnd\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412305 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412262    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-podres\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412505 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412333    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-lib-modules\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412505 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412383    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56mnd\" (UniqueName: \"kubernetes.io/projected/99d1aedb-1bac-4475-8b63-2370a104d55b-kube-api-access-56mnd\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412505 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412454    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-proc\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412505 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412475    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-podres\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412505 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412491    2582 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-sys\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412805 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412516    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-lib-modules\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412805 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412554    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-sys\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.412805 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.412612    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/99d1aedb-1bac-4475-8b63-2370a104d55b-proc\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.422025 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.421987    2582 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56mnd\" (UniqueName: \"kubernetes.io/projected/99d1aedb-1bac-4475-8b63-2370a104d55b-kube-api-access-56mnd\") pod \"perf-node-gather-daemonset-mmh9r\" (UID: \"99d1aedb-1bac-4475-8b63-2370a104d55b\") " pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.511657 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.511542    2582 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:02.713991 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:02.713942    2582 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"]
Jul 23 18:32:02.718581 ip-10-0-139-151 kubenswrapper[2582]: W0723 18:32:02.718538    2582 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod99d1aedb_1bac_4475_8b63_2370a104d55b.slice/crio-c3cf000a5fe96ef9356e56288c149e9285f28ba8601118e1c90651341960dc98 WatchSource:0}: Error finding container c3cf000a5fe96ef9356e56288c149e9285f28ba8601118e1c90651341960dc98: Status 404 returned error can't find the container with id c3cf000a5fe96ef9356e56288c149e9285f28ba8601118e1c90651341960dc98
Jul 23 18:32:03.448362 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:03.448320    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r" event={"ID":"99d1aedb-1bac-4475-8b63-2370a104d55b","Type":"ContainerStarted","Data":"301230df0652960ce79ddedcbd3293311228ecc39e2be2348e18c1508a15bdd5"}
Jul 23 18:32:03.448832 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:03.448393    2582 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r" event={"ID":"99d1aedb-1bac-4475-8b63-2370a104d55b","Type":"ContainerStarted","Data":"c3cf000a5fe96ef9356e56288c149e9285f28ba8601118e1c90651341960dc98"}
Jul 23 18:32:03.448832 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:03.448582    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:03.466841 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:03.466770    2582 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r" podStartSLOduration=1.466747149 podStartE2EDuration="1.466747149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 18:32:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 18:32:03.464827179 +0000 UTC m=+2818.304335128" watchObservedRunningTime="2026-07-23 18:32:03.466747149 +0000 UTC m=+2818.306255100"
Jul 23 18:32:04.447594 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:04.447563    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qrgxp_960e922b-95fb-4745-9093-b48be740ce28/dns/0.log"
Jul 23 18:32:04.468115 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:04.468086    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qrgxp_960e922b-95fb-4745-9093-b48be740ce28/kube-rbac-proxy/0.log"
Jul 23 18:32:04.560599 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:04.560568    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-l79t9_26784fee-febb-4712-9906-14c5a6d9a352/dns-node-resolver/0.log"
Jul 23 18:32:05.202680 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:05.202630    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rb9ft_24b62c73-baa3-4159-9529-58aa59ebf21d/node-ca/0.log"
Jul 23 18:32:06.195323 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:06.195292    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h72j8_8fefef8d-63df-4f11-84a0-483681b242cb/istio-proxy/0.log"
Jul 23 18:32:06.434213 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:06.434176    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6c5d7974-k9nj7_f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2/kube-auth-proxy/0.log"
Jul 23 18:32:06.489333 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:06.489252    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5sn9_d2eb6cca-0902-4227-8a43-b223e53fd3b2/istio-proxy/0.log"
Jul 23 18:32:06.525163 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:06.525130    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5sn9_d2eb6cca-0902-4227-8a43-b223e53fd3b2/istio-proxy/1.log"
Jul 23 18:32:07.298338 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:07.298309    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b7hbt_5d7340a2-0a7a-4786-ac9c-5068e0dc7d8b/serve-healthcheck-canary/0.log"
Jul 23 18:32:07.970894 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:07.970861    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xtzsf_9a34bd01-1bac-4775-9c86-c2e589b4c3b7/kube-rbac-proxy/0.log"
Jul 23 18:32:07.991235 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:07.991195    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xtzsf_9a34bd01-1bac-4775-9c86-c2e589b4c3b7/exporter/0.log"
Jul 23 18:32:08.012499 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:08.012469    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-xtzsf_9a34bd01-1bac-4775-9c86-c2e589b4c3b7/extractor/0.log"
Jul 23 18:32:09.470759 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:09.470699    2582 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-v5cbl/perf-node-gather-daemonset-mmh9r"
Jul 23 18:32:10.680360 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:10.680332    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-9xks5_a9e4e79b-ddd7-40b3-bdf8-2bebbd11c785/server/0.log"
Jul 23 18:32:10.968324 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:10.968246    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547c58d5d9-hbh8z_5dcac08f-5065-4dfa-bd0a-d5ccba92c976/manager/0.log"
Jul 23 18:32:10.989782 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:10.989754    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-547c58d5d9-hbh8z_5dcac08f-5065-4dfa-bd0a-d5ccba92c976/copy-manifests/0.log"
Jul 23 18:32:12.487477 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:12.487437    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-754bfdc447-n4bvg_d01399fd-01d1-45a8-b785-10da9da185cf/manager/0.log"
Jul 23 18:32:12.524005 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:12.523967    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-gstdx_59aee851-756c-4d70-9855-90ea91354016/openshift-lws-operator/0.log"
Jul 23 18:32:13.104297 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:13.104256    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h72j8_8fefef8d-63df-4f11-84a0-483681b242cb/istio-proxy/0.log"
Jul 23 18:32:13.332320 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:13.332290    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6c5d7974-k9nj7_f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2/kube-auth-proxy/0.log"
Jul 23 18:32:13.372598 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:13.372511    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5sn9_d2eb6cca-0902-4227-8a43-b223e53fd3b2/istio-proxy/0.log"
Jul 23 18:32:13.413226 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:13.413197    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5sn9_d2eb6cca-0902-4227-8a43-b223e53fd3b2/istio-proxy/1.log"
Jul 23 18:32:14.664747 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:14.664657    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4h72j8_8fefef8d-63df-4f11-84a0-483681b242cb/istio-proxy/0.log"
Jul 23 18:32:14.893675 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:14.893640    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5b6c5d7974-k9nj7_f3d89cb6-c9b2-4f7c-99c5-068e2b30fbd2/kube-auth-proxy/0.log"
Jul 23 18:32:14.934019 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:14.933926    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5sn9_d2eb6cca-0902-4227-8a43-b223e53fd3b2/istio-proxy/0.log"
Jul 23 18:32:14.972638 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:14.972599    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-m5sn9_d2eb6cca-0902-4227-8a43-b223e53fd3b2/istio-proxy/1.log"
Jul 23 18:32:19.432386 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.432360    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-76gc6_ec002160-229b-4bc8-a52f-12354c87c023/kube-multus-additional-cni-plugins/0.log"
Jul 23 18:32:19.454540 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.454514    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-76gc6_ec002160-229b-4bc8-a52f-12354c87c023/egress-router-binary-copy/0.log"
Jul 23 18:32:19.476355 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.476329    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-76gc6_ec002160-229b-4bc8-a52f-12354c87c023/cni-plugins/0.log"
Jul 23 18:32:19.498614 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.498586    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-76gc6_ec002160-229b-4bc8-a52f-12354c87c023/bond-cni-plugin/0.log"
Jul 23 18:32:19.519348 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.519320    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-76gc6_ec002160-229b-4bc8-a52f-12354c87c023/routeoverride-cni/0.log"
Jul 23 18:32:19.541838 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.541813    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-76gc6_ec002160-229b-4bc8-a52f-12354c87c023/whereabouts-cni-bincopy/0.log"
Jul 23 18:32:19.562297 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.562268    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-76gc6_ec002160-229b-4bc8-a52f-12354c87c023/whereabouts-cni/0.log"
Jul 23 18:32:19.812302 ip-10-0-139-151 kubenswrapper[2582]: I0723 18:32:19.812272    2582 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rcf5c_88012f00-e808-4fe4-9fbe-f0825cfd0ff7/kube-multus/0.log"